Well, I am not a minimalist living with five books artistically displayed on a shelf, by any means! I still have two large and two small bookcases and my goal is to get the books down to the two big bookcases. I also have a large book collection in my office at work; in my field older books are still really valuable and I like being able to lay my hands on something without hiking down the hall to the office collection. And let's not talk about my Kindle/Nook apps!
My criteria may not work for you, because I am not (mostly) a rereader. Basically, I only hold onto books I expect to use again, or a small collection with sentimental value. Right now I have a lot of books that are just here temporarily, because I let the Amazon used books and used book store purchases get out of hand. But since I tend to only buy hard-to-find titles, especially ones that aren't available as e-books, I don't want to get rid of them until I read them, because I can't get these items at the library. Once they are gone, I intend to get rid of the smaller bookcases, and only have a shelf of "To Be Read" at any one time.
So for me, keepers are:
1. Cookbooks that I have used at least once or expect to use again
2. A small collection of books by people I know or with personal inscriptions
3. A small collection of religious books: I have my christening Bible, third-grade Bible, Confirmation Bible, a hymnbook that belonged to my grandmother, and my actual reading Bible now that my confirmation Bible is no longer repairable
4. A few favorite popular fiction books and all the books by one fantasy author I particularly love--these are my comfort books that I do go back to over and over.
5. A few reference books that theoretically I could get rid of, but either I still use them or they have sentimental value (like the dictionary and collected works of Shakespeare I used in college), or a couple of books on decorating where I want the pictures for reference.
I have moved enough over the last 20 years to have not piled up a lot of books that fall outside these categories--last time around, I got rid of cookbooks that no longer reflected how I cook, or that I didn't like, reference books for hobbies I've gotten bored with, or books that had accumulated somehow that I wasn't interested in reading or rereading.
If you are trying to decide what to keep and what to give away, perhaps thinking about what you would want to have right at hand and what you could wait a few days on from the library would help?