Fellow humid climate-dweller here, commiserating with your situation. We're at 80%+ much of the year, 65% or lower feels positively dry. I live in an 'old' apartment building in Hong Kong, built cheaply in the mid-20th century, with natural cooling via airflow and generally not much mold though not well built. Now we've all got aircon units and I'll use mine if necessary since my flat was subdivided from a larger one and heat can get trapped. Still when we have the hallway door open, air from the rooftops whooshes down and clears out heat straight away.
This year we had several weeks of unrelenting heavy rain. I got mold inside all my wooden furniture, and under my kitchen sink. It would've been worse but I'd placed open jars of salts inside which absorbed quite a bit of water. Chunky salts like kosher and himalayan, left over from a botched bath bomb project. Also lots of bamboo charcoal which helped dehumidify and deodorize too. Frequently use a HEPA filter for air purification as well. I tend to keep windows sealed while out during the day (open if home, but shaded) and sealed at night with dehumidifier on due to the pests in my neighbourhood, and have an exhaust fan running 24/7. I try not to have aircon and dehumidifier running at the same time, it uses up so much energy but sometimes can't be helped. Generally I try to stay in tune with weather outdoors and as long as there's a fan, find a dry room more comfortable even in hot weather and don't need to cool things down.
It sounds like your house may be made of various materials, some of which could be exacerbating the problem and there's only so much you can do. Interesting to hear about interior temperature changes as sources of dampness. Best of luck!