If you enjoy it, then do it. The Hubs volunteered at our church for several years building houses for low-income people (sort of like Habitat for Humanity). He even went on a 10 day trip to Pass Christian, Mississippi to help rebuild after Katrina.
In the process, he learned many valuable skills that he has used around our house to save us some major cash. He learned how to retile a bathroom and shower, which we are going to utilize to update our house to sell. This year our hot water heater broke and he was able to install a new one himself, saving us thousands of dollars. Without volunteering and learning how to build a house from scratch, this wouldn't have been possible.
I also went on a 2 week mission trip to Africa to distribute malaria nets and work in a mobile eye clinic. I can give you a vision test, if you want one. :) I found that helping people, even when you acquire new skills, is much more worthwhile than the skills you acquire in the process. I think both The Hubs and I found both experiences incredibly humbling and increased our gratitude quotient by a lot.
One of our goals after gaining FI is to be able to do more volunteering like this. I want to take my kids on a similar trip when they are old enough to handle the emotional part of the experience.