Okay, here is our situation and what we did. I have no idea if it would work where you are, but I suspect many markets in the US are more lucrative than the ones we are in.
1. We purchased a home in a smaller popular summer resort town across from the beach with 10% down. The value is 1/3 the beachfront value but it is on a cul de sac adjoining a beautiful public beach and has a lake view. We legally suited it (above ground suite). Vacation rentals are legal here as well. We suited, soundproofed, painted and furnished it (CL) in a one-month period. We advertised on CL and through a vacation rental site and did our own webpage. We rent it out by the week at very high rates in the summer and we also use it as our go to vacation destination (three weeks already this summer). In the winter one suite is rented out by the month at a much lower rate on a term lease. We keep the other suite for us as it is also a ski town. This property nets $5000/yr after all expenses and provides a free place to stay for our family. This year for the first time we will also rent out the upper suite which will increase the profit by net $6000.
We do not have property managers. I do the bookings, which is not much work imo (approx. 12 a summer). I do the spring and fall cleaning (with help). I organize the long-term tenants. I have family in town who I pay to organize the cleaning and respond to any issues that arise. We have not had a problem with damage or theft in four years of renting out. We rent to families with children almost exclusively and some of them have been coming for three years in a row. Each year I take some of the profits and make improvements. This year will be some landscaping and painting. I find it fun to improve the place and make things better for us and for visitors.
2. We bought a triplex in a bigger city with 20% down that is in a very nice, upscale, walkable area. It was in very rough shape when we bought it (flophouse-like with tenants to match - they moved out as I don't think they wanted to live next to the landlord) and we have been working hard to renovate and improve it. We improved the first suite that needed the least work right away (in about a week - painting and furnishings) and rented it out partially furnished to three very nice, quiet, university students. They are staying another year at least. We lived in the next suite and left the third unoccupied (it needed the most work - pretty horrible and dirty actually). We have been working to improve both of these suites and finished the second suite, furnished it nicely (under $1000 from CL) and rented it for Sept 1 two days ago. The combined rents from the two units cover the mortgage/expenses.
We are moving to the third unit which is not finished the last week of August. This unit does not have a finished kitchen and still needs work - we will live with it that way for a bit. It needs a ton of finishing work. That said - it is essentially free. It has 4 beds and 2 baths and laundry and it will be lovely when done. Eventually we will paint the outside and landscape the property and it will match the rest of the neighbourhood. At that point we may use one of the units as an executive rental at a much higher rate.
You are right that you need to do the math. You need to understand the vacancy rates, cleaning costs (we charge a fee that doesn't quite cover the cost), and any management costs. Our place is more profitable because we manage it ourselves, but we need to put the time in for this. Do not just talk to the real estate agent. Talk to other owners who are doing this in the same place. I would say multi-family properties (if properly organized and soundproofed) are generally more profitable.