Well yeah, there's a downside - your home is tax free! If the market goes up that's huge!
But if it goes down...
IMO owning the least house possible is still the better idea - lower council tax, lower heating, lower maintenance. I've serially made the 'mistake' of owning smaller or otherwise less expensive places, and in some ways it has been 'a mistake'... but no when you look at the cost of debt, the cost of maintenance, it's fine.
Over medium-long term the stock market should do better. However there is a lot of risk with stocks. If you have enough home and don't ever plan on 'needing' more, 'having' more is risk without a reward you need. It sounds like you're way way over already if you can live on 11k.
You can of course put money into the SIPP and keep it in a risk-free fund (not 100%, in North America it'd be called a 'money market' fund, there must be something similar? - where it is safe as cash but does return at least a little interest), and drip it into ETF shares every month. Statistically 'all in' is better though.
Basically you have to tally risk vs reward. You won't be getting off the housing ladder entirely, but its fluctuations will affect you less.