Based on personal experience of going from an average blue collar school district (average school in Mass, so I don't even know how awful an actually bad school would be...) to an affluent top of the state district, I think school district is hugely important. I know a lot of people think that school rankings are flawed because of economic advantages but IMO that's a feature, not a bug.
As a parent you don't care about whether the rankings reflect teaching quality, you care about whether they affect outcomes for your child. From that perspective, I think being surrounded by classmates from "advantaged" households is a huge benefit in terms of fostering ambition and teaching them that they have to compete for good outcomes. It also helps that the teachers were better paid but honestly the problem with the mediocre school district wasn't the teachers, it was a negative & infectious environment where kids didn't care about school.
I know everyone likes to think that parents can offset this but in my experience that was 100% not the case; the few kids who tried to exhibit genuine intellectual curiosity were bullied and eventually squandered their potential in the blue collar town. They did better than the meatheads, but not by much. On the flip side, in the affluent school district you had maybe 50 kids who each individually would have been the valedictorian at a worse school; to me that is more important life experience if your kid later wants to compete for a job at Google or a PHD program at Harvard...
This is going to vary SO MUCH based on location and kid and even year.
Some kids are going to do better as a "big fish in a small pond" because they are doing well.
Some kids will thrive with a lot of "high quality competitive kids", and some will shut down.
Some schools are able to group kids so that highly motivated kids aren't dealing with bullying. (I had to deal with it, but that's because the vast majority of kids in my home town were not into studying.) But then, the good thing about being bullied for 10 years is that I was pretty much PROPELLED out of that town. Other people who had a more enjoyable life are still there. Where there are no jobs.
So, in our case now, we are in a "middling" school. There are enough interested kids that my kid is challenged and excited about school. We specifically transferred out of a school that was 95% poor and EL, because we didn't want him to be the *only* (or one of the few) interested kids. With a larger group, the school does pull-outs for reading, match, etc. so that advanced kids stay advanced, and the kids who are behind get extra help.
We specifically avoided the upper-middle class school next door. Just not my jam. A little research showed me that kids of my demographic do AS WELL as all the kids at the other school or schools, and they get to do it while learning about people who are different from themselves.
That said, even WITHIN a school, there are differences. When kid #1 went to school, there were a LOT of kids. We have open transfers in our district. Meaning, you have to register at your home school, but you can request a transfer to a different school. You only get one choice, really (though there is one charter school that does its own lottery, and that's pretty much the yuppie school for all the people trying to escape brown people. No I'm not biased or anything. But I've heard actual conversations from parents at that school.)
Anyway, this is where we opted for the "middling" school - it was almost equal distance from our home, just in the other direction (less than a mile away). We were turned off by "School #1". Well, School #1 has the GATE program AND it's less than a mile from the school we chose. So the school we chose loses a lot of UMC families to School #1. Except that year. Because: so many kindergartners that year that my kid had 3-4 kids in his class (not grade, class) whose older siblings were at School #1 and they DIDN'T get in on a sibling transfer (which is top of the list, behind only teachers at the school). These kids didn't get in on a sibling transfer until second grade (so parents had 2 kids at 2 different schools, though some opted to come back to the home school).
The side benefit, for us, is that there are a LOT of highly motivated kids in my son's grate, and by the time the GATE test rolled around, most of us decided to stay rather than transfer.
However, the grade behind him is much smaller. Has been since kindergarten - only 2 kinder classes vs my son's year that had 3, and they were packed at 29.
So, that grade is weak. It's smaller. A lot more families were able to transfer out right from the start. For the few that stayed - their kids were getting bullied, so they left too. Now, there are literally 3 kids in the whole grade that are GATE/ advanced, and they are getting bullied. One of the parents fought for a decade to get parents to stay in the home school and said "well, this is making her miserable - do I keep her here because of my own feelings?"
Right now, my younger son's class is looking pretty good. But enrollment is declining in the district. I already know of 1 or 2 families within a block of the school who didn't even give it a chance.
If kid #2 starts having issues with bullying, I wouldn't hesitate to transfer him. Not even a second.
TLDR:
Schools vary by year, by district. Kids vary too. You have to know your kid. What you choose in kindergarten doesn't have to stay that way.