Typically, an offer to purchase should include a contingency on being able to secure satisfactory financing.  If the lending falls through, you are entitled to your deposit back.

If the money accompanied a P&S, the terms of that document, and not the offer, will dictate under what circumstances you are entitled to recover your deposit.

You should have an attorney review the documents.  If you are talkign about a small deposit (like $500), it may jsut be worth it to walk away and chalk it u p to a lesson learned - make sure you work with an attorney on any future real estate purchase.