You have some good advice already about how to get ducks in an order if leaving is the best outcome.
So, I'm going to focus on the job itself. You had a good job. Almost all of the people you have to work with are fantastic. You didn't give much detail about what happened there other than a horrible new boss, really inappropriate stuff he is doing, and an unresponsive HR. It's fine (and probably best) to not share more on an internet message board. If the new boss suddenly left, would you be sure you want to stay in your job? Or are you struggling anyway with other aspects and thinking it might be best to leave?
Here is my take on what I read and what I am reading between the lines, FWIW:
The first, and most important, thing to realize here is that YOU HAVE THE POWER. That is because you don't have to work, which means you don't have to put up with this bullshit. Technically he is "above" you on the food chain, but in reality you have more power than he does. Period. You can afford to lose your job, and he most likely cannot. You win. You have the power.
If he was reporting to you instead, what would you say/write to him whenever he crosses the line? Keep it professional and appropriate. Plan it out. Fight back directly to his face a bit with professional words that make it clear you do not have to put up with poor and inappropriate treatment . . . because you will not be bullied. No one has the right to speak to you that way, sir! He might be nasty to you because he wants you to quit; if that is the case, then if you quit, he wins. Do you want him to win? No. (Just don't say the F word to his face . . . keep it clean.)
I hope you are documenting everything daily in writing and printing it out to keep in a separate file. Every tiny instance you feel crosses a line, no matter how unsure you are how it would sound to someone else, document it with the date. You may already be doing this, and, if so, good for you! I've once worked in the gas-lit sea of microaggressions, with some overt aggression thrown in, and you might also be in that situation. It's hard, in that situation, to not doubt yourself on some of the more "minor" inappropriate things. Don't doubt yourself, because you have the power here. Be sure you print out a written record of the first time you went to HR along with any response that they gave (their written response, if their was one, or your recollection of their verbal response, if that is all there was.) Do all of this on work time.
Then, decide what you want to do. Things I have done or seen other people do:
1) Minimize your contact with him in every way possible. I know you can't totally avoid him, but minimize as much as you can for now.
2) When he crosses the line, stand up to the bully/harasser and make it clear that it stops, now, period. Lather, rinse, repeat.
3) Take work time for your health! You have a doctor's appointment in the middle of a work day . . . with a therapist! He or she will give you more ideas and help you work through all of these issues. See that person at least once a week (2X per week if you think it would be helpful.) This is coded as sick time. You are not crazy, but we all benefit from an impartial & trained third party listening and lending ideas. Honestly, I had one colleague have a therapist certify him going on disability leave due to bullying-related work mental stress . . . obviously that is an extreme measure, but it was an effective strategy for him.
4) More self care during the day . . . once a week have a massage or facial, paint your toes, retail therapy at the local thrift shop, whatever it takes and whatever works for you. Do this "during lunch", or code the time as personal time, if your company has that. Pamper Momma! Because if Momma ain't happy, then ain't nobody happy!
5) One-on-one, over the phone or over lunch, tell as many co-workers as possible what is happening on the down low. See what they say. Some might have similar problems with him, which only increases your power further, or some might have really good tips on what to do. It can't hurt.
6) If the company is large enough to have an "Office of Institutional Equity" of something like that, then file a formal complaint with them. File it as sexual harassment whether or not any of his behavior is sexual, because at a minimum he is
creating a hostile enough work environment to make you want to quit. I've done it, and it sucks, but it's an important step. Bear in mind that their job is entirely to keep the company from being sued, and that they probably will not rule in a way that you think is completely satisfactory, but at that point you are pretty protected . . . if your boss fires you after that, it is more clearly retaliation. He will then likely be told he can't fire you, and in a bigger company there is a good chance they will change your reporting relationship. If he is doing it to you, then he has done it to others and will do it again in the future . . . this is doing your part to give those future ladies more leverage in seeking recourse. It's not just about you. Social justice and equity! Power to the people!
7) Escalate it again through a different pathway, perhaps his boss or his boss's boss. Usually his boss's boss is the best pathway, since that person likely did not hire him directly and is not as invested in backing him up. This I have also done, and it is scary, I know. Use your documentation to help guide that conversation. Keep it professional. Talk about how much you love your job and how you want to work in the best interests of the company. You have to guess at who or the upper ups will be more sympathetic to your plight. HR is useless; go to his bread and butter. This kind of end run will make him hate you, but it will also get his behavior on their radar. Do you really care if you get let go? At this point they'd probably have to give you some kind of severance package.
Again, you have the power. No sir, he will not treat you like this.
Good luck!
