I didn't read all this thread, but just want to comment that it has been noted by educators that middle school-age girls will read books with both boys and girls as lead characters, while boys will generally only read books about boys. So, the "sexism" starts early. For what it's worth.
Why is this sexism ?
My opinion is that world has become so weird now that you can't be who you are because you get judged on the basis of your racial background or gender. I think bizarrely the people doing the judging are the ones calling other people of being sexist or racist.
Sexism = evaluating people differently based on their sex.
Making a choice to read books based on the sex of the protagonist, by this definition, is sexist.
I hope this answers your question.
I think there are some fine lines here. Authors, perhaps more than any other creators of media (games, movies, tv) DO have targeted audiences. Genres are HUGE in publishing. Heck, to get published traditionally you need to be able to fully describe your targeted audience to agents, editors, publishers, etc. and explain why your book will appeal to them.
Many books are specifically targeted to middle-aged women, or middle-aged men, or young women 12-16, etc. So the author is generally writing "to" someone, and if you're not in that "to" group, it might not be a natural fit for you to appreciate and enjoy the book, although it's certainly possible.
Twlight was clearly targeted to young women. That doesn't mean that young men, or older men, etc., can't enjoy it, but they certainly are not the audience being addressed.
I think we can all be more mindful of stretching our interests and reading books in genres we haven't read before, but I don't see my wife reading 'pages of descriptions of military weapons" Tom Clancy books anytime soon, and it doesn't necessarily make her "sexist" to not enjoy extremely masculine-themed books like his.
That being said, I do think books like Harry Potter have broad appeal to folks in lots of different demographics. Some books just have wider appeal than do others. Most romance/erotica books are geared to women. It's just the way it is. It doesn't mean a guy can't pick one up and enjoy it, but again the author is not writing for him. The author is writing to a specific audience and including the tropes and expectations that women expect to find in the story. It's more financially effective than trying to write a romance book that would equally appeal to men and women.