My boss is actually transitioning to a developer role right now because he's tired of the 60+ hour weeks. He's moving to a different team, under a different IT director. The spot he's taking is one that was already open, and they're hiring to replace him. He wants to downshift permanently, so he's okay with the fact that this might be his last role until retirement, whenever that is. That's really a best-case scenario.
I think it's unlikely you'll be able to transition all the way down to senior developer on your own team, both because they don't have a replacement groomed for you and because your juniors are conditioned to look to you, and they aren't going to stop without a lot of effort on the part of you and your boss.
It's unlikely your boss is going to want to lose you if the team is so heavily weighted towards junior developers. When is budgeting time? If you want to convince them to hire someone new to help you, then you need to be having this conversation before budgeting conversations start. I'd make sure to have a lot of documentation on the kinds of tasks you do and the amount of time they take, so that they can see this is more than a one-person job. In some cases, you may be able to recommend that tasks X and Y be delegated to someone else/some other group.
I would definitely have a backup plan, just in case.