For me, it wasn't about net worth, but about having cash in the bank and having worst-case scenario plans. For the first 10 years of our marriage, we kept virtually no cash savings (we did have modest retirement accounts). Our emergency fund was an open line of credit on the house (remember those days? when housing prices only went up? and treating your house like and ATM seemed like a fine idea?). Between home equity and retirement accounts, our net worth was about 400k, so we were doing OK.
After the economy faltered, I had an entirely different perspective and no longer considered a line of credit a good source of emergency funds, so we began building our cash savings. Once we had enough to get through a year with neither of us working, I started to feel safe. I feel even safer now that I have planned what we would do if he lost his job, if I lost my job, if we both lost our jobs. Worst case scenario is we sell our house, use the equity to buy a smaller house in the same school district, put saving for retirement and college on hold, cut expenses, and we could get by on about 20% of our current income. That's what makes me feel safe.