Withdrawals from your HSA for medical expenses for your dependents are qualified. You can confirm this by reading the section on qualified distributions in the instructions for Line 15 of Form 8889 at irs.gov (
https://www.irs.gov/pub/irs-pdf/i8889.pdf, page 6, rightmost column).
Given your family situation, your kids may or may not have been your dependents over the past several years. As I read it, you would be able to use medical expenses for your kids in those tax years in which you claimed them as dependents on your tax return. Whether or not you claimed them as dependents on your tax return is probably determined by your divorce decree or the equivalent agreement in your state. (In years when your ex-husband claimed the kids, it's unclear to me whether the kids' medical expenses would qualify, but I would think not.)
The other rule that may matter but probably doesn't is that the expenses in question have to have been incurred after you opened the HSA. But that should be an easy bar to clear.
Back in 2016 and 2017 I had SELF+DEPENDENTS HDHP and had HSA account open and contributed family max. The plan covered me and my two kids. In 2018 their dad remarried and kids are now covered by his wife's insurance. I contribute max to my self HSA annually.
This is fine, assuming that the years in which your kids were covered on your plan and the years you contributed the family max were the same years, and that the years they were not on your plan that you contributed the self-only max.
I think I heard a while back that if HSA account was open when kids were covered by the plan then their medical expenses can be covered by HSA even if they are no longer covered by that plan. Is that true? I can't seem to find any information on this.
No, that is irrelevant. What matters is if they are your dependent. See my general comments above.
Also, in 2019 my company closed that HSA account and moved us to a different HSA account. So at that time, I transferred all of the old HSA account to Fidelity. Would this be a problem if I in fact can use HSA to cover their expenses.
Totally irrelevant, and thus not a problem.
I do not use any funds in HSA account to pay out of pocket medical costs so accounts (Fidelity and current company account) are fully invested but I may need to take some money out of HSA for some large expenses in 2022-2023. I currently have about $10K of my medical expenses over 2017-2021 that I can pull out but also there would be close $12K of kids' expenses that I can document and support from 2019-2021, all after they left my plan.
Again, what matters is if they were your dependent during the years they incurred the expenses. See above.