I wish I had better insight to offer. Personally the main concerns that I would think of are exactly the ones you mentioned in your first post, except that
@Villanelle's testimony illustrated the possible problems for kids much better than we could imagine.
That said, whether bad stuff would happen is IMHO a dice roll... the kind best handled by thoughtful exploration (as you're already doing) and and wise situational responses. There's no way to be sure beforehand which choice is better, but my guess is that you go for what you want.
I do think that if you live in an exceptionally wealthy area, it's important to give children experiences of other areas, not just verbal teaching. It's tricky enough and important enough that IMHO you should budget time for such things.
If you have habits, enthusiasms, etc that put you in touch with different social strata, that would help, whether they be camping buddies, hiking clubs, Habitat for Humanity builds. Are things that would get you out of the neighborhood and give the kids familarity with other levels of society a part of your life that would continue if you moved to the place with the best view?
I have known several children from wealthy backgrounds. They varied widely in what they know about life, and how they explored the world. Some developed wise values, and used their familiarity with the upper crust in creative ways that greatly increased their positive impact as adults. So there's potential upside for the kids, as well as downside.
On average, it's unclear that money helped most of them a lot, but I don't think it hurt the ones I've known. I think whether it hurt or helped particular cases was very individual. So I suggest broadening the kids' experience as thoughtfully as possible, and going for your dream view.