What to do about hiring an employee from my former employer and my non-compete agreement?

It depends on two things:
1) What EXACTLY does the contract say? If it says you can't employ anyone from the former employer, you would clearly be breaching it. If it does say that you cannot "solicit" or "encourage" them to leave, then if they reached out to you (and not vice versa), you would be legally ok, but--
2) If the former employer litigious--do you think they'd sue in this situation? If you think they would--maybe they hold a grudge? maybe they feel this is disloyal somehow and should not be allowed? etc.--then bear in mind that they could  certainly at least file a lawsuit and force you to defend the suit. Even if you'd ultimately win--because on the facts as applied to the terms of the contract, you have not breached it--you'd have the cost, time, and distraction of dealing with a lawsuit.
3) And bear in mind that if you do violate the agreement, or at least what you do is close enough to a violation that a court could, if you were sued, get it "wrong" and find against you (except in the most clear-cut cases, you can't be guaranteed of winning), you could be ordered to not employ this person and have to let them go, even if doing so were disruptive and costly to your business; you could also have to pay monetary compensation.
That you are asking the question implies you have doubts about hiring this person. When in doubt, don't: there are other qualified candidates out there, and you don't need, especially as a new business, to be inviting trouble or litigation.