I would be worried about the ethics of it. When you start stealing (and yes it is stealing) to save money you should start reconsidering your life choices. If you really want to do it, walk inside and ask the owner if he minds. If no one cares, you will have no problem getting one to let you do it and then there is no moral issue involved.
Here's a thought. I work downtown in a mid-sized city and I pass a couple of dumpsters every day on the way into the office. If you are recycling most of your trash then "borrowing" someone elses dumpster (which never gets close to full) is a possible solution. It may not be 100% ethical, but its akin to taking a bucket of water from someones lake. No one cares in the long term.
I wouldn't worry about the ethics of it. But more the legality of it. A gum wrapper or a plastic water bottle is probably fine. But a big bag of garbage might get you into trouble, if caught.
Oh, it's a gray area to be sure and as I've said I've never done it. Wait, no, I take that back. Buddy of mine years ago had some major project and at the end of the day he had a whole truckload of garbage to deal with, well more than what his trash service would haul off. So we used the half empty dumpster where he worked - without permission. Holy $%^&, were my ethics compromised and I wasn't even aware???? ;-)
Oh, please. No harm, no foul.
This is not a slippery slope to hacking banks and stealing from ATMs all over Manhattan. This is a lot closer to jaywalking than robbery. But yeah, I'll gladly cede to the authors this is not legal. Just as jaywalking is not legal.
So here is perhaps a better idea. Your neighbor has a trash service, right? Split the cost with him. Provided there is nothing in the contract that says you can't. There,
problem challenge solved, conscience clear.