A few thoughts:
I went from Ph.D to postdoc, to a second postdoc--spent more than a decade surrounded by Ph.Ds (some of whom were dumber than shite). After the 2nd postdoc, I took a job as a lecturer at a large public university (4 yr, B.S., M.S. granting)--teaching undergrads, some of whom are fresh out of high school--quite a change in surroundings!
It really is a job that takes a while to grow into. At first, preparing lectures takes a LONG time. My first semester, I signed up for two new classes, and spent pretty much every minute of my time preparing. Figure 1hr of lecture = 10hrs preparation. HOWEVER, after you teach it the first time, it is vastly, vastly easier and less time consuming thereafter. I still update/improve my lectures as time goes on, but at this point, I have everything down to a science and the job seems like easy money--especially given the hours I put in, and the fact that I have TAs to do my mundane grading, and people to do all my printing and copying. I am also enjoying it much more than previously. I love my field and enjoy seeing the students (some of them) get excited about it.
I have definitely improved at this job over time, become a better lecturer, and have learned to have a more relaxed attitude. My evaluations have steadily improved, to the point that last semester in one class 90% of them gave me the highest score. The students greatly appreciate open-book, open-note exams. I am not big on memorization, and stress that knowing how to find info is equally important to having info stuffed in your head.
Having a UNION makes a great difference. In many places, higher ed is just another form of big business and administrators often view students and faculty as an inconvenience--a necessary evil to be dealt with as they please. Taking time away from the golf course, cocktail parties, and schmoozing with wealthy donors, is of course their biggest annoyance. Their salaries continue to rise way out or proportion to everything else--some folks (not you) are making a killing off of this business! In any case, the union has won some important pay raises, benefits, etc. We also are free (as we should be!) to teach the classes as we see fit. (Some idiot administrator at another university was telling faculty not to talk about climate change because he is a corrupt, idiotic POS. That would never fly at my workplace)
One of the best things of the job is that I really have no boss per se (don't want to p1ss off the dept chair or dean obviously), and am good at avoiding my co-workers by an unusual schedule. My office is a nice oasis, where I can work on other, extra-curricular interests while not teaching.
In any case, give it time, try to get hired at a place with a union (if not ORGANIZE one!), and learn to relax and realize that most of your students will never have the level of interest you do. A mentor of mine once stated: "I don't consider it my job to judge the quality of students we have here. It is my job to teach the students we have to the best of my ability." Also, try to have some compassion--realize that many of the students (even at community college) are racking up crushing debt, some of them come from very disadvantaged backgrounds, and an alarming number of college students are homeless and suffer food insecurity--'Murica!