I'd recommend you start by clarifying your goals: what do you want to end up with?
I did this when I downsized last summer. I made a pile for each of my adult kids--not every photo I had of them, but enough for them to have a full record of infancy to adulthood--individual photos, pics with siblings and treasured friends, special occasions. I did not feel the need to give them all 12 photos taken a a birthday party-- I picked a good representative photo and tossed the rest. I gave each kid their pile and told them they were free to keep them, scan them, frame them or toss them as they saw fit, but I was done storing them.
I kept photos I loved and put them in frames or a photo album which lives on the coffee table, where family and visitors can enjoy them--people now act like it's a retro treat to handle a photo album, it's strange!
Everything else got tossed.
When my grandparents died there were boxes of slides (Tulips in Holland! Canals in Venice!) and photos of people we didn't know. Here are twelve strangers in front of a tree, with "Topanga, 1943" written on the back. Nobody knew who they were or why the photo was saved. My mother did not want to keep them and could not stand the "bad juju" of throwing them away, so we donated them and let the charity shop disperse them to the winds of eBay or the landfill.
My method worked for my goals (everyone having a record of their growth and family and happy times). Your method may need to be different if your goals or situation are different.