To be fair, it's not the only "key to the benefit". It's still completely tax advantaged*. For example, say you contribute $1000 pretax. It's even better than contributing $1000 to a 401k, because there's no FICA taken out. It's straight $1000 saved with zero tax burden. Then you go to the doctor, and get a bill for $1000 (how convenient!). You reimburse yourself (or pay the medical bill with the provided debit card linked to your HSA account, or whatever), withdraw the $1000 tax free, and pay the bill with the $1000. At no point did you pay any taxes. That's still a hell of a nice deal! Even if you never invest the money, and just keep paying your medical bills with it, you're still gaining a tremendous tax advantage over time.
*Depending on the state you live in.