I can't speak to the loan officer issue but I'm an attorney, often working on environmental compliance issues and I think your existing skill set is very valuable to the right employer. What you were doing is really essential because the liability exposure for companies dealing with hazardous materials is very high. Here's a link to read
http://money.usnews.com/careers/best-jobs/compliance-officer. That's an awesome job with above average pay, upward mobility, and a below average unemployment rate all in a job sector with strong growth. Corporations, municipalities, the military, universities, shipping companies -- EVERYONE needs an environmental compliance officer.
I'm very surprised that you've had little success finding employment in that area and I suggest you do some additional research to identify what you may have overlooked. Have you tried identifying relevant associations, like this
http://www.naep.org/. If you have a college degree, call the career services department at your college and get their help.
Network your buns off. You're considering leaving a great job that has high barriers to entry (thus, fewer people competing for positions) for a job that has low barriers to entry and is very prone to boom and bust economic cycles. That alone should make you pause.