not pointed out so far is that these speaker fees for past presidents are not out of line with speaker fees for other well-known and sought-after people on the corporate speaker circuit (e.g. current/past execs of fortune 500 companies, efficiency/productivity gurus, lifestyle coaches, even celebs). It's also in-line with what big-name entertainers will command to play similar private events
It's not uncommon for any of these to command 6-figure fees. Also worth noting to this isn't really for just ~2 hours of their time, as preparing a proper seminar can take a week or more and involve several aids (so 40-100 work hours -- still incredible pay but ot quite as obscene as the '$200k for 2 hours work!' headlines make them out to be).
So in a very real way, past-presidents command high speaking fees because those are market rates, and investment banks like Goldman Sachs routinely shell out $1MM for 'talent' at theircorporate events.