Normally, the ignition lock is designed to prevent the key from being removed in any position except completely "off." However, if the cylinder is worn out enough that might no longer be the case (on my 1996 Ford Ranger, I can remove the key in any position, including while driving).
It could just be that you didn't turn the cylinder all the way to "off" before taking the key out. Try turning it "more off" (with or without putting the key back in first) and see if that helps.
Either that, or maybe you wired it up wrong when you replaced it.