First of all, congratulations on getting out of debt! Welcome to the forums!
A lot depends on what you interest rates on the mortage are and how much you make. If your annual income (after all deductions and everything) puts you the 25% tax bracket or higher (assuming you are in the US) then it is almost certainly better to first maximize the amount of money you put in tax-deferred accounts, like a 401K, Traditional IRA, or HSA. The money you will put into those accounts you will have to invest someplace, so I'll +1 Phoebe's recommendation of starting out with index mutual funds. If you are not in the 25% tax bracket, then you'd probably be better off maxing out any tax-now-but-not-later accounts like a Roth IRA.
Once you've done that the question becomes what to do with the rest: pay off mortgage or invest in a regular account. It really depends on what your mortgage interest rate is, what you expect the stock market to return on average, and how comfortable you are with risk. If you mortgage is at 4% for example, paying extra on the mortgage is saving you money at a 4% rate every year, guaranteed. If you expect the stock market to return 7% on average (which is not at all unreasonable), then it's mathematically superior to invest in the stock market instead of paying off the mortgage. However, stock investments are not guaranteed, so they are inherently riskier than paying off the mortgage. In addition, paying off the mortgage has its own psychological benefits, especially once you own your house outright. So the question becomes not one of what's best but one of which is better for you, it may be worth it to you to loose the extra 3% / year the stock market would give you to reduce your risk and get your house paid off faster. PMI can be annoying, it does not necessarily tip the scales one way or the other in this debate. You'd have to figure out what your "real" mortgage rate is (including PMI) and then see if it's worth it to you to give up the potential stock gains to pay that down.
The best advice I can give is to figure out what you want to do for these first few decisions, go with it, and then as you go spend time learning about investment theory, diversified portfolios, and asset allocation. You will see people discussing these topics all the time on here, and there's no way to know ahead of time which way is best. It's all a circus act of trying to balance investment returns with an acceptable level of risk. Basic advice though is to not leave all of your eggs in one basket. That's what "diversification" is all about. Index funds are a way to own many bits of stocks instead of putting a lot of money in a few stocks, which increases diversification. Investing in a mix of stocks, bonds, REITs, Lending Club, etc. instead of just putting everything in stocks increases diversification across the different kinds of investments you have. The problem of course is that most other forms of investments will not have nearly the returns of stocks / Lending Club, so you have to balance diversification with putting you money where it can actually make more money.
Anyway, I think you should:
1) first take maximum advantage of any tax-advantaged investment accounts you can get
2) decide where to put any extra money: mortgage or additional investments
3) take some time here and there to learn about additional options as you go
Hope this is helpful!