The rules vary by jurisdiction, and you should retain counsel for an opinion on the legal claims that may be available in this situation. I can provide some general information but it is only general information that cannot be relied on.
Generally speaking, every residential lease is read as containing a covenant by the landlord to provide a habitable property. Breach of that covenant is actionable as breach of contract. Whether the presence of mold in the apartment violates that covenant would have to be researched, but it might. Once the landlord learns about unhabitable conditions, he may then be in breach of his covenant, even if he acts quickly to fix the problem. See, e.g., Knight v. Hallsthammar, 29 Cal.3d 46 (Ca Sup Ct 1981) (noting that "where, as here, a landlord has notice of alleged uninhabitable conditions not caused by the tenants themselves, a landlord's breach of the implied warranty of habitability exists whether or not he has had a 'reasonable' time to repair").
As mentioned, I can only provide general information, not advice on your specific situation. It sounds like you would benefit from a consultation with a lawyer to determine whether the case is worth pursuing.