I honestly wouldn't mind if this home halved in value since I am never moving.
I think the above statement is key in understanding why some people will say owning a home is a bad "not an" investment. If you never plan on moving, some people myself included do not see a primary residence as an investment. I consider it an asset that has the ability to keep up with inflation, but not an investment. Currently, I have the same outlook of never moving. If I change my mind and sell I would still have to live somewhere and the money from the sale of my house would likely be rolled into a new place. I don't see buying a home as a negative, just not an investment.
On the other hand if someone bought a home to live in, fix it up and flip, or rent it out, I would consider that as an investment.
Respectfully, the flaw in this position is the analysis needs to include the imputed rents. That seems to be the omission that JL Collins, some Bogleheads and some MMMers are making here.
This obvious statement: If I buy an investment for $100,000 and two decades later I sell it for $50,000, that change in value doesn't on its own mean the investment was bad.
Similarly, if I buy an investment for $50,000 and over two decades it grows in value to $100,000, that change doesn't mean the investment was good.
The investor needs to also include the cash flows (in or out) she or he experienced over the two decades.
So, the interest income if a bond.
The dividends if a stock.
The cash flows if a direct real estate investment.
The cash flows effects if a home you own and occupy.
Put all this stuff into a spreadsheet. Calculate the internal rate of return--or better yet, the net present value. And voila, you'll then know whether the investment was roughly good or bad.