I have a similar situation. Old farm house on a field stone foundation, first floor level roughly 2' above the ground.
We blew insulation into the stud cavities in the walls, however the rim joist and portion of rock foundation extending above grade had zero insulation. Also like yours, lots of little gaps and things just not fitting real tight, leaking air, entry points for mice, etc. I had a foam insulation contractor come in and spray foam from underside of floors, on inside face of rim joists, then on rock walls to about 2' below grade to seal everything up tight and to get some insulation in that area.
They also embedded a heavy screen wire in the foam so that mice couldn't chew through the foam and still get in. This isn't 100% foolproof, but it did help. Seems like mice go with the territory in an old country home.
Can't really quantify the cost vs savings, but you can tell the place is a lot more weather tight, and the floors are warmer in the winter. I'd recommend it.
Note - I doubt you could do a real good job like this do it yourself, as it requires some specialized spray foam equipment.