I should say that I'm thinking I'd like to spend $500 or less.
Also, I've been to all 50 states and most of Eastern Canada so going for the sake of seeing a place isn't as exciting anymore. It's about interesting activities - hiking, history, art, science, etc. I know - I'm a travel snob. :P
So you want to spend $500 or less on 4-day romantic weekend? I'm not sure what kind of romantic trip you expect to take on that budget.
That is only $125 per day, which includes all lodging/meals/activities/transportation costs!
That's not enough $$ to fly, so it should be within a ~200 mile radius of NYC, unless you want to spend much of your time driving.
That doesn't leave many frugal locations to choose from.
The only thing I can come up with is Pennsylvania... either the Pocono Mountains or the Lancaster/York areas.
Even then, you'll have to stay at the Budget Inn and eat at Applebee's and McDonald's to stay within budget.
Doesn't sound very romantic to me.