I also recommend a Mac Mini, allowing them to use their existing display, keyboard, mouse, & printer. Unlike an all-in-one or laptop, individual components can be replaced inexpensively if they die.
For web browsing, ebooks, & limited messaging, the elderly do very well with a tablet like the iPad that allows them to zoom text to whatever size works for them, plus its touch surface is more intuitive to operate than using a mouse. An iPad no longer requires a computer for setup & backup.