Hey, I'm in the process of buying a house two states away! It kind of sucks, though. Especially for us, since we found Dear Wife & I weren't on the same page we thought we were for the first 2-3 trips. Hrumph. But, uh, it's totally possible. People do it all the time. But especially since you already know the city you're going back to.
Now as to your question - gosh, that's a lot of money on a house. I live in the Great Lakes Region, where you could buy a half a block of homes for $575k. I know things work differently out on the coasts, but that doesn't change the fact that it's nearly 5x your salary. Ouch! Holy shit!
First of all, your monthly payment is not $2,365. That number is for suckers, because only suckers would buy a 30-year mortgage. Seriously. A 30-year mortgage is just paying the bank less principal each month so you can have a higher interest rate. It is the worst invention of all time for personal finance, and the single biggest obstacle to FIRE. It is going to take you forever to pay that thing off. So to summarize,
DO NOT GET A THIRTY YEAR MORTGAGE.
If you can't afford the 15-year mortgage, you can't afford the house. It's really that simple.
And should you use the taxable brokerage money? Hell, no! That's the nest egg of your Freedom Ticket. If you put that money in a house, you'll never see it again. Instead, leave it in investments where it will get bigger & bigger - so one day you can choose not to work.
But lastly, consider the house you're buying. Houses you live in are not investments. It's a money sucking sinkhole. Taxes. Repairs. Utilities. Even the home equity is your hard earned money that refuses to generate more money. Because there's absolutely no guarantee you'll ever sell it for more than you buy it for. When in my 20s, an older co-worker once advised me that "a rich person has a big house and a small car." No. The rich person (or at least the person getting rich) has a small house and a small car. Because then all of their extra money goes into investments. And *that's* how you get rich.