I don't think you're going to get away with a ground-source heat pump for hot water heating - that's a screwy enough situation that if someone did make a unit for it, you wouldn't want to know how much it cost.
Rock means going straight down, which is also expensive - you're basically paying someone to come drill a well, then run pipe down and fill it in.
Assuming you're planning to live there a long while, I'd look hard at doing an air source heat pump/air conditioner, paired with a natural gas furnace for the dead of winter. You can use the heat pump when the air is warmer out, and you'd have to figure out the crossover point for air source heat pump vs natural gas furnace in terms of cost of heating, though if you put solar on, you may value using the heat pump down lower than is optimal.
A modern air source heat pump works down to ~-10F or so, so the old advice that you can't use them anywhere below freezing isn't accurate anymore.
I don't know if it would be cheaper/more effective to rig in a forced air gas furnace, or to put a gas boiler in for your heating system. But if you're going to get a new air conditioner, the cost of adding an air source heat pump is quite low - it's just an air conditioner with a few extra valves and wires.