<.<
Uh, private pilot here. So, yeah. Not the cheapest hobby I've ever had. That said, I got my license in grad school, so you can do it cheaply enough.
A few thoughts/comments/etc, in response to various things in this thread:
The "If it flies, floats, or f****" comment is close, but "renting" isn't the best option. A good flying club is. It's cheaper than renting, by a lot, because you're expected to put in time and labor as well as money, and it's not for profit - the point of the club is to maintain the fleet and have some spare cash for upgrading things (we're in the process of getting rid of the vacuum gyros and replacing them with either Aspen units or Garmin G5s, because for about 2 rebuilds on a gyro AI, you can replace the whole thing with a unit with battery backup). I'm part of a flying club that has 7 airplanes, ~105 members. We show up to help with the plane washes, chipping ice out of the taxiway in the winter, people help with oil changes, and for the avionics upgrades, there are a few retired guys who are happy to show up and basically dismantle the entire interior of an airplane before the A&P goes at the installation. The cost for all of this is $60/mo plus $60 "use it or lose it" against flying. The goal here is to encourage people to fly each month, at least. If I fly, that $60 goes against my flight time - I only pay it if I don't fly at all. And the winter is reasonable in terms of having a few months lumped together, because January out here is pretty much "Nope" for flying.
In exchange for time/labor/etc, my flight costs are quite low. A 172 is $72/hr wet, a 182 is around $110/hr wet (that covers fuel as well). The 182s are a slightly better deal for long trips.
Now, in terms of traveling - if you do travel a bit, and are flexible on your schedules (and/or have your instrument rating), the cost for a family of 3 or 4 to fly Cessna Air is comparable to commercial.
For a concrete example here: I live around Boise. I have a brother in Phoenix. I have a wife and daughter (there is one part of the "flies, floats, f****" bit I disagree with...).
For the three of us to fly from Boise to Phoenix round trip (Southwest, who I prefer, and isn't particularly expensive) is around $1700 - and we're looking at 5-6 hours total travel time, which practically means 7-8 hours (with the 2 hours TSA nonsense). Plus, for me, an hour to get to the commercial airport instead of 20 minutes to get to the local one.
For me to fly that in a 182, at a 144kt 75% cruise, I'm looking at 4.5 hours in flight (nonstop), though I'd probably make a potty stop halfway there. And, for... oh, about 10 hours on the hobbs, $1160 (for the nice traveling 182). Wet.
Driving, I'm looking at 15+ hours, which is not a one day trip with a toddler, so add at least a hotel. It's certainly cheaper, but I do prefer flying.
And I can carry a family of 4, which is even more expensive commercial ($2300). I can also fly from general aviation airport to general aviation airport, which saves a lot of time over flying from commercial to commercial (there are little airports pretty much everywhere).
I'm not going to weigh in on a private license for the particular OP in question. If the observation about other debt and such is accurate, well, probably not a good idea now.
But in terms of being a useful way to travel around, if you're willing to be flexible with regards to weather, for 2+ people, it's faster than commercial for a lot of trips, and for 4 people? Hard to beat it.
And, admittedly, I live in a place where I have access to a cheap flying club. And the planes are focused a bit more on back country flying and cheap ownership (no retracts). But I consider it well worth the cost in time and money. Being able to fly yourself around is awesome. Being able to fly into little back country strips is pretty cool. And, I mean, I'll even go flying to play Ingress ("Planegressing") if there's a good excuse and I need to fly for the month. I flew with someone a few weeks ago to mess around with the concept. We're going to do it more, because it's a great excuse to fly.
In terms of up front costs, $12k is absurd. You should be able to do it for a lot less. For my flying club, if you figure flight training in the 152 ($60/hr) for 50h (minimum is 40h but most people get their license around 45-50), 20h of an instructor at $40/hr, and 6 months of base fees, you're looking at a hair over $4k. Plus ground study materials and a headset and such, $4500-$5k is totally doable at my club. And then $1500/yr, with 12 hours of 152 time included in that. I think my total for the private was closer to $5k, but that was renting at much higher hourly rate because I hadn't discovered the joys of flying clubs back then.
If your goal is just to be in the air (instead of traveling)? The recommendation for sailplanes is spot on. I've got 1.5 hours in a glider (properly fun intro flight), and it was the most fun flying I've done. Feeling the air, constantly turning or maneuvering, hunting thermals, thermaling with hawks... absolutely awesome. Not a great way to get from point A to point B, but a super cheap way to be in the air and flying.
Is it extreme Mustachian? Certainly not. That said, I'm not nearly as hardcore as plenty of other people here - I keep the lifestyle expansion down, but we're willing to spend money on things we value (and traveling is one of those things). But a lot of the costs being thrown around are simply wrong, if you can find a good flying club.