We upsized by doing this (facepunch) so why couldn't you do it in reverse? Twenty years ago, we started in a one-bedroom apartment, then moved to a 1200 square foot house five blocks away, then moved into a 2400 square foot house in between the apartment and the first house, all in the same neighborhood.
When I think about downsizing, I'd want to stay in the neighborhood. We've got deep roots here now, wonderful friends and a strong sense of community, close to downtown, public transportation, a few blocks from the elementary and middle schools, walkable...yeah, they'll need to move me out of this neighborhood on a gurney!
If you're satisfied with your neighborhood, I think downsizing in the same street is a great way to go. If you could literally carry your furniture and personal items from one house to the next, it would also be a cheap way to move.
I vote for yes!