Personally I wouldn't. The tax benefits, high CoC returns, tax free income due to expense deductions, principal pay down, appreciation (in my area it's huge) are just too good to pass up. Frankly IME RE investing is better for FIRE than index investing, as it requires less capital for higher return, and you get tax benefits to boot.
My returns in RE just from cash flow alone are far higher than I could safely get in the market. Add to that appreciation and such and it's ridiculous. And one day the mortgages are paid off and my income increases that much more. And I don't need to worry about stock dips, running out of money, etc.
That said, part of my strategy will involve having several hundred thousand in index funds that I could live off alone OR...use on occasion for emergencies. Since my rental income far exceeds my expenses in retirement, even this is overkill. But I like safety. Plus there is nothing wrong with working for a few more years at my income, it's a huge opportunity cost for me to stop working, so I might as well get money while I can.
If I get to the point where I can't get a job or have my desired target savings, I will likely stop working. That said, with a career change I could see myself working to 45, which I would consider to be a reasonable time to retire as well, since I would have worked a long time at that point. Currently 33 planning on retiring at 40.