What can I do if I’m in a non-legal partnership and I want this man out of my home?

Does he pay rent to you?
If he doesn't, he's just your guest and you can make a guest leave at any time. If he doesn't go when you demand that he does, you can file a legal action called an "action for ejectment" in civil court to remove him. "Ejectment" is basically eviction for non-tenants (guests) and you will get a judgment against him that can be enforced by sheriff's officers or court officers/constables, who will physically remove him if neceessary. If he is doing damage, you can seek to get it on an "emergent" basis (an expedited basis: think "urgent" or "emergency"). Ejectment actions are more complex in some ways than evicting a tenant, and emergent actions are more complex yet: you are advised to retain an attorney to do this for you. (The lawyer can also help act as a buffer between this man and you; and the lawyer can also look into whether some sort of protective order is also advisable.)
If he is paying rent, he is a tenant. That means you can evict him like other tenants if:
1) He fails to pay rent;
2) For damaging your property and disorderly conduct; and/or
3) If there is no written lease, on 30 days notice (if there's a written lease, when it expires).
You can get forms and instructions from county court--this is a more of a "standard" or common procedure than ejectment--but again, you are advised to let a lawyer help you: if you make a procedural error, the case would be dismissed and you'd have to start over, and from what you  write, you would not want to lose the time.