My sister's second husband's first wife (say it slowly) did something along these lines before/during their divorce. Very similar. When my sister married him, she discovered exactly why his ex might have done such a thing.
I think he's on his third wife now...
Adding another layer... When my sister started dating him, she was in the process of buying a house. The ex had the kids show her where it was. She found one for rent just around the corner and moved in. She would open the door and send the kids to my sister's house at dinner time. Worse, still, she would drop them off when my sister wasn't home and take off. The poor kids would sit on the porch with their coloring books, waiting for my sister to get home. Turns out all of the adults, my sister included, deserved each other, but those poor kids! (I am not touch with any of his kids, but I have heard they all survived into adulthood.)