Many loan companies REQUIRE you to use escrow to cover annual insurance and tax payments - the loan companies don't want to take on a property as collateral only to find the user stops payments, the house burns down, AND the owner has no insurance.
If I could go without paying escrow, and pay taxes and insurance directly, I would do that in a heartbeat. Go for it!