Under what conditions is a car salesman entitled to keep a portion of a deposit if the buyer backs out of a deal?

This is a used car lot, correct?  First, as I am sure that you now realize, leaving without getting anything in writing can lead to serious problems down the road.   What you do have is an oral contract for the purchase of the car but one needs to check if this type of contract violates the statute of frauds in your state for this type of transaction.  Under the statute of frauds it states that certain types of contracts must be in writing to be valid.  This could be to your benefit depending.  It is my understanding that car dealers are not required by law to give a 3 day right of cancellation of a contract.  That would be at their discretion.  he fact that no such discussion regarding cancellation or fees was had between you may also work to your advantage.  Sue him in small claims court and file a complaint with the state attorney general's office.  Good luck.