Can I break a lease if I moved into a rented house and discovered after I had drank the water, it was contaminated with coliform bacteria?

In every lease there is an implied warranty of habitability which requires the landlord to maintain the premises in a habitable condition by complying with state and local housing codes.  The contaminated water is a health issue which constitutes a breach of the implied warranty of habitability.  When there is a breach of the implied warranty of habitability, the tenant notifies the landlord and the landlord is required to respond within a reasonable time by making the necessary repairs.  When the landlord fails to respond within a reasonable time by making the necessary repairs, the tenant has the following options:  The tenant can make the repairs and deduct the cost from the rent (this would be impracticable in your situation) or the tenant can move out and terminate the obligation to pay rent for the balance of the term of the lease or if the tenant stays on the premises, the tenant can withhold rent and defend against eviction.  Another alternative is to sue the landlord for breach of the implied warranty of habitability.  You can also contact your local housing code inspector, who can bring an enforcement action against the landlord for housing code violations.

A separate issue here is if you required medical treatment due to drinking the contaminated water, your landlord is liable for your medical costs.  Prior to filing a lawsuit against the landlord to recover your medical expenses, it may be possible to settle the case with the landlord's insurance carriier.  When you complete your medical treatment and are released by the doctor, obtain your medical bills, medical reports and documentation of any wage loss.  Your personal injury claim filed with the landlord's insurance carrier should include these items.  Compensation for the medical bills is straight reimbursement.  The medical reports will document the nature and extent of your injury and will be used to determine compensation for pain and suffering, which is an amount in addition to the medical bills.  Compensation for wage loss is straight reimbursement.  If the personal injury claim is settled with the landlord's insurance carrier, NO lawsuit is filed.  If you are dissatisfied with settlement offers from the landlord's insurance carrier regarding your personal injury claim, reject the settlement offers and file your lawsuit against the landlord based on premises liability.  If the case is NOT settled with the landlord's insurance carrier, you will need to file your lawsuit prior to the expiration of the applicable statute of limitations or you will lose your rights forever in the matter.