The Subaru works great as your family car so why trade down to an older, less spacious, considerably less useful for any sort of outing with the kids or camping unless you desperately need the money you might save.
You have a 55-mile trip daily and know those bells and whistles give you pleasure every day - so absolutely hold on to what you already have and enjoy it.
I'm all for optimizing and if you happen to be in a financial pinch I'd bite the bullet and do whatever is necessary to lower costs and expenses.
I get why DH doesn't want to sell a paid-off car, but nope, I wouldn't give up my Subaru unless I had a real good reason to do so.
Isn't owing $16k on the Subaru "a real good reason to do so"? Is that what people mean when they talk about the forum getting soft?
I have an Outback, without the bells and whistles, and I have three kids. I will admit that it's pretty nice to be able to put the stroller in the back, but other than that, I don't see why the prius couldn't work. I always thought the trunk area of a Prius was pretty spacious, but maybe I'm mistaken there. How old are your kids? Could you use the Element for the trips that require extra gear, like camping, etc?
I own a prius (2013 i think), and used to own an outback (2008). The trunk of the prius is decent, but not great. Especially with the slope of the window it can be difficult to fit large rectangular items there, like a stroller on it's side, or boxes (or a dog crate). Large (ish), flat things work fine. The outback has way, way more room. And OP has a newer design than mine was, I think it got even bigger/taller. Now, whether that space is "worth the cost" to keep a larger, inefficient car I don't know. There are also some transactional costs to changing vehicles. When I've done the math the small savings in gas usually it's worth it to change cars just for that reason, only when evaluating vehicles you are changing anyway. But I always buy cash so haven't considered loan costs, and I drive less than op.
As for the question, it does seem a bit non-optimal to have two larger, less efficient cars. We have two kids, and haul larger tings occasionally; like lumber, furniture, camping stuff, grandparents (from out of country so they don't have a car). So we have a prius for around town driving and my commute, and honda pilot for larger items, fully loaded beach/camping trips and third row seats for guests. It works pretty well for our use. And both cars of course bought used 3-4 years old, so didn't cost that much, minimal maintenance, and should last a looong time.
Unfortunately, IMO the best option is to switch your husband's car for a prius! :D (Don't tell him I said that)
I remember testing an Element, and didn't care for it. I has surprisingly little space, goofy layout, and really not efficient or optimal in any way (from a hyperrational, engineer perspective at least.) no offense..
edit: also can you really get $20k for the outback?? I sold one with 100k miles, and got $6k I think! Though it was older than yours
and bought a prius with 30k miles for $14k..