Yes, it's probably best to have life insurance for both of you.
Which country do you live in? In some countries, it's common for life insurance companies to offer Joint-First-to-Die term insurance which can be 5% than each of you buying your own coverage.
Also, if you're planning for FI within 20 years then you probably wouldn't need lifetime protection. Just ask for a 10-Year or 20-Year term product. Term life insurance is so cheap that you could request $600,000 of term insurance for a lower premium than $300,000 of whole life insurance. At your young ages, buying whole life insurance isn't *too* expensive though, but Term should still be much cheaper.
Also, ask whether you have the option to convert your term into whole life in the future. Many insurance companies allow you to buy Term insurance now and then switch to permanent insurance on request. Your permanent insurance will be cheaper if you lock in at a younger age, but since this is an Early Retirement website you may be better off just buying the Term insurance and then amassing a huge pile of cash so that life insurance is no longer needed in 20 years.