Late May can vary from shorts and a T-shirt to medium weight jacket.
If I were you, I'd skip the car rental and stay in Montreal. Quebec City is a pretty long drive and would be a bit far for a day trip (that's a 6+ hour driving day). Also, Montreal is a terrible city to drive in, so if you can avoid it, you're better off taking transit and maybe a cab now and then. The metro system is quite good, though you'll probably want to take a cab to/from the airport.
For your anniversary dinner, if you want to do a fancy restaurant, I'd suggest L'Express on St. Denis, which is a super old-school French bistro. It's not cheap, but it's not stratospheric for a fancy place, and the food is excellent. Slightly less expensive and less intimate would be L'Academie which is like 4 doors down and a nice BYOB French/Italian.
For cheap places to stay, I don't have any specific suggestions, but I'd look on AirBnB. Montreal is a huge student city, and there will be a lot of vacant apartments for short term rental. For a quieter neighborhood that's right downtown, and will have lots of rentals, I'd look at the "McGill ghetto" which is basically bordered by Sherbrooke, Pine, and St. Urbain. Here for example is a 1 bedroom apartment for $75/night
https://www.airbnb.com/rooms/2302578?checkin=05%2F22%2F2014&checkout=05%2F26%2F2014&guests=2&s=xLCXFor places to visit that are a bit off the beaten path, if you like beer, the St. Ambroise brewery has a nice outdoor bar and I think you could tour the brewery.