Can my employer listen to my office conversations without my consent?

Do you mean he stood near you and listened to the conversation? That is perfectly legal. There is no law against "eavesdropping" unless a person is trespassing to be in the area from which he is listening; presumably, your boss had the right to be wherever he was in the office. Also, there is no protectible privacy right in conversations you have with a coworker at work.

Or do you mean he listened in on your office phone calls? That would be legal if there was some--any--notice to you, such as in a employee handbook, that your employer could monitor phone calls made over the office phone system. (Though if he was listening to phone conversations made over the office phone to/with people outside the office, such as customers, they, too, should have been provided notice that the call may be monitored.) The phone system belongs to the employer, and the employer may monitor its use.

The employer could even listen to "your end" of a phone call you take at work on your cell phone--i.e. listen to what you say. The only thing the employer could not due is actually in some way "wiretap" or listen into both ends of a conversation you have over your own cell phone with someone outside the office.