I am sure I am repeating some from the others, but FWIW I will put my 2 cents in.
Below is the link to the investment order thread that simplifies how to maximize your money.
https://forum.mrmoneymustache.com/investor-alley/investment-order/Phones - $109 (2 cell phones) - Definitely look into other prepaid carrier...Republic Wireless, MintSim, This could be much less.
Preschool - $410 As someone else mentioned does your husband have an FSA?
Actually I think the rest of your set spending is pretty good, Especially in HCOL area. Check with your car insurance to see what your deductibles are. Raising to the max will lower it. You do not give the age of your vehicles, but do they need full coverage?
Variable Spending
Groceries - $1200 - I think everyone has beaten this one to death.
Restaurants - $350 - Same as above (maybe look at groupon and find other deals)
Kid Stuff - $200 Looking for the free or cheap activities. Buying second hand clothes, etc.
Babysitter - $75 Like others have said get creative with cheaper dates and/or work out a deal with friends to swap babysitting.
Medical - $200- Does your husband have an FSA or HSA... Again take advantage of pretax dollars.
My money - $100 - Maybe find a youtube yoga class, meetup groups, learn how to make your favorite coffee/latte at home, find free entertainment
Husbands money - $100- I am guessing alot of this may go to work lunches? Cut his hair at home?
Personal Care - $100 I am guessing the 3 year old will be potty trained here soon and this will probably go down. Learn to shop sales. When your shampoo goes on sale, buy enough until it will be on sail again
Fuel and Parking - $150 - How far does your husband live from work? Would it be worth moving to be closer? Combine trips?
Family fun - $50 Again look for some free opportunities - Groupon, parks, family game nights.
Marriage Therapy - $125-250 HSA/FSA?
Household Maintenance/Updates - $150 (yard supplies, paint, fixing this or that, chicken feed)
Household Decor & Goods - $150 (decor, cleaning products, toilet paper, etc) - Looking for sales
Housekeeper 1x/mo - $85 (mostly covered by cash from stuff I sell as we minimize things) - The 9 and 6 year old can definitely clean. Three year olds love to do what mom is doing.
Yearly Expenses/Sinking Funds - $400/mo (Christmas, CSA membership, car registration, COSTCO membership, Birthdays/Gifts, Zoo membership, etc)
Savings to Emergency Fund - $400/mo (Sadly, sometimes a portion of this gets taken from the budget for overspending in other categories)
Like others have said there are plenty of things to cut and minimize. There are a few low hanging fruit phones and housekeeper, but you did not get to this level of spending overnight so it will take practice and building new habits to become more frugal. I think buy paying yourself first i.e. automatic savings 401k, IRA, EF then spending after will help.
You said making it a game or challenge. Find ways to include the kids. Be sure they are learning the value of a dollar. Visuals are always helpful. You could use a thermometer picture to fill in as the EF gets to the $20,000. If you do some sort of challenge putting stars or stickers on the calendar for each day of success.
Be sure to come back to share your progress.