What do you drive now? What are docking fees compared to the insurance, taxes, registration, maintenance, extra gas and insurance on a 4 Runner? The tow vehicle could potentially cost more than the boat itself. Be careful! The thought of purchasing a vehicle just to tow a boat makes me cringe.
We live 15 minutes from a beautiful lake and 10 minutes from an very nice class 1-2 river. Our nicest spring and fall weekends get filled quickly. Realistically, we get on the water 3-4 times a year (but hoping to do more once retired!).
As others have, I suggest starting small with a used kayak(s) or canoe(s). They can be found cheap on craigslist and sold later for nearly the same or even more a few years later. If you truly find yourself on the water as much as you think you will, only then upgrade. When you do upgrade, do it slowly, maybe every 5-10 years, and enjoy the process. This will allow for waning interest which may happen after the first year or two on the water. Yes, you can get efficient at it, but loading everything up, driving to the lake, launching, recovering, drying everything out, covering, annual maintenance, etc. does involve some level of hassle. After the initial year or two you might decide it just not worth the effort. Don't find this out after investing tens of thousands in a boat and tow vehicle.
Also, you didn't mention the age of your children. But if they are young, they may be pre-sports and other activities. If so, you may feel like you have tons of free time on the weekends, but once they start participating in these types of activities time gets shorter and boating is not typically something you do for an hour or two when you have to tow. Generally, you need the majority of the day. Just something to think about.