I spent three months at a deep field camp in Antarctica (multiple times!) with virtually nothing to do. Work a shift, then unlimited free time until your next shift. No tv, no internet, no radio, no trees or rocks or topography, almost no other people. Just barren flat whiteness for a thousand miles in every direction.
I was looking forward to it, at first. My life had been so crazy hectic for so long, before that, that I welcomed the change of pace. No obligations, no stress, all the sleep you want, I thought it would be great. It was not great. I did get to read lots of books, but I found myself becoming obsessive about inane things in order to provide structure to my daily routine. I was fanatical about keeping a journal, and reading books on a specific schedule, and practicing specific skills every day.
I learned that I need to participate in something in order to be happy. Like I just need other people around to interact with, like to be part of a community. I needed variety in my experiences, and without any natural obligations I felt compelled to create my own. Fortunately, I was pretty successful at creating my own.
So I don't fear early retirement. I'll find a way to keep busy, and I suspect you could too.