As I understand it (I've spoken to holiday management companies about possibly renting out a house as a holiday home), if you have a desirable holiday rental your clients will often return year after year for the same week: after three years you should have a solid list of "repeaters" for high season weeks who will probably book fairly early - sometimes booking for the following year immediately on finishing the current year's holiday. That's in the UK, though, so your case might be different - but if you can get regular clients, that would be good. Perhaps try to find out what your clients think of your place and whether there is anything that needs doing to encourage them to come back?
What put me off the holiday rental idea was that there are now fairly strict rules in the UK on taking children out of school during term-time, which means that the rental "season" for a family home is shorter than it used to be - no families with school age children will want to rent in September, for instance, which means that your target market is much more limited - families with children under school age, or couples, or dog walkers, etc.
The benefit of vacant weeks is that they are available to use yourself.