The fact that the bank created the estimate twice and you didn't contest either estimate is probably all the bank needs to demonstrate that they twice made estimates in good faith and you didn't argue with either estimate.
Thing is, **one month** after sending me the early 2014 letter saying they had overestimated the taxes and my escrow was going down (and enclosing the $10k check), they received and paid a tax bill three times higher than what they had paid the previous year... and didn't tell me, and didn't adjust my escrow back up or request the $10k back, until getting the NEXT year's huge tax bill a year later! In other words they knew within a month that they'd made a huge mistake on the 2014 taxes, but chose not to tell me, and not to collect enough escrow payments to cover the taxes for 2014 or 2015, for another year!
And then when I called them they said (1) that I somehow should have known they had made a mistake back in early 2014 (them: "You should have known what your taxes were and that our escrow wasn't going to be enough." Me: "But YOU don't even seem to know what my taxes are, and it's YOUR job to set the escrow correctly!"), and (2) that they were legally obligated to collect the deficiency over a 12-month period, no longer and no shorter--which is a lie (it's true that under RESPA it can't be shorter than 12 months, but there is no restriction on how long it can be; they have the option of collecting it over 24, 36, however many months, instead of jacking up the mortgage payment so high most owners would lose the property).
As you and others have suggested, I'm going to ask to pay my own taxes and insurance to avoid this kind of screwup in the future.
Also, I talked to a colleague who knows this area of law and he told me to report it to my state banking department and the federal Consumer Financial Protection Bureau. Done.