I agree, the less someone drives, in general the worse they get at driving. But I wouldn't worry about it, because let's look at the potential consequences. To use hypothetical numbers, let's say for every halving of your driving, you increase the chance of you causing a wreck by 10% when you do drive. If we assume you're an average driver, half of the wrecks would be caused by you and the other half by the other driver, so the actual increase in wrecks would be 5%. And since you've halved your driving, you increase your chance of those wrecks over the same amount of driving before you cut your driving in half by 2.5%.
So bottom line, even though you've increased your chance of causing a wreck when you drive by 10%, you've decreased your overall chance of getting in a wreck by 47.5%. I'll take those odds.
And keep in mind every time you go out to practice driving, you are only increasing your chance of getting in an accident.