There's a part in the profile settings where you can see what your peak times are for looking at the site, and which boards are your favorites. If you really want to cut down your time here, find out why you are here at specific times (for me, it's during slow periods during my workday) and find a different thing to do at that time. If you have a specific board you're at all the time, think about why that's your favorite and then try DOING more of it rather than reading about it.
Personally, I'm OK with the amount of time I spend here. It's better than half the other things I could be doing.