Assuming you have a good rate on your mortgage, I wouldn't bother paying that off early (unless you have PMI, which if you do, keep paying more to get rid of that).
With your "life fund," you are basically saving for things you want...like upgrades to the house and vacation? If so, I don't see any downside to having that. My wife and I basically have an "All Other Spending" category in our budget. We decided to combine a lot of categories into one (household essentials, entertainment, travel, etc). So now if we cut back spending, we can roll this budget over every month to allow it to grow and then take a vacation or buy whatever we want. We feel it is very important to be able to travel, so we like having a budget that allows us to do that.
As a couple, you need to decide what is important in your life and set up your budget accordingly.