If I found a roach infestation 2 days before I was due to move in, what are my rights?

You almost certainly cannot break the lease at this point. A landlord does have an obligation to provide a habitable apartment (called the "implied warranty of habitability"), and a roach infestation, if severe enough, can violate this obligation. However, before terminating a lease early would be justified, the landlord must first be given a chance to correct or remediate the problem. Only if the landlord has failed to act after notice (preferably written, sent some way you can prove receipt) and a reasonable chance to take action and exterminate (and a reasonable opportunity could be a few weeks, since an exterminator must be scheduled) might you be justified in terminating your lease. And even then, termination is the most severe or serious remedy to which you might be entitled, and it's not always justified, even in cases of a violation of the implied warranty of habitability--sometimes, the correct remedy is monetary compensation for the tenant, or the tenant being allowed to arrange to fix (exterminate) the problem him/herself, then deduct the cost from the rent.

First give your landlord notice and a chance to take action. If he/she does not, then consult with a landlord-tenant attorney to discuss your options.