Instead, if it were to work in a way analogous to the IRA world's regular Backdoor Roth, the entire rollover from your 401(k) after-tax subaccount would be treated as if it were a rollover from your various 401(k) subaccounts on a pro rata basis, even though it was in fact a rollover of only your after-tax account.
I think this is another case where the 401(k) world differs from the IRA world more than we would expect it to.
In the 401(k) world, the distinction between funding sources is tracked within your account by your custodian,
and none of those details are reported to the IRS.
In the IRA world, it's the opposite: the IRS tracks the breakdown between pre-tax and after-tax money (via your yearly filings of Form 8606),
while the custodian sees it all as one pot of money.
In the above paragraphs, the parts in black I'm sure of, while the parts in grey I'm less-sure of. It was difficult to verify in my own Vanguard account, and certainly different custodians could vary in their tracking/reporting, but it didn't seem like there was any way to specify that an IRA contribution was intended to be "non-deductible". Most tellingly, Form 8606 and 1040 allow you to decide at filing time how much of your IRA contribution you want to deduct (it doesn't have to be the maximum allowed), so even if there was a way to tell your custodian "this contribution is deductible", that would be superseded and made irrelevant by whatever you ended up telling the IRS.
So in the case of a pro-rata partial IRA conversion, the IRA is seen as a single pot that simply becomes smaller when the conversion is rolled out. It's an external tally of Form 8606s that defines and updates the mix inside that pot.
Whereas with a pro-rata partial 401(k) conversion, the 401(k) custodian
better pull the money from the correct sub-pots, because as far as I know, there's no other method to know how to update the status of the remaining mix.
In bearkat's example, it seems clear that the conversion was not
treated as the 99% after-tax money that it was intended to be. In that case, there should still be (a lot of) after-tax money available to do another after-tax conversion. In the IRA world, there would be, because after the first conversion, the calculations in Form 8606 would determine what the new available after-tax amount is. But in the 401(k) world, where the custodian is simply showing no money left in the after-tax pot, I don't know of any other method by which a subsequent conversion of after-tax money could be made.
Which, IMO, makes the situation particularly bad for bearkat. It would be one thing if Vanguard had incorrectly pulled out pre-tax money, and then treated it as pre-tax money; that's getting taxed earlier than you'd like (and penalized), but at least it's just getting taxed once. But pulling out
after-tax money and treating it as pre-tax money is taxing that money early AND taxing it twice.