I wouldn't simply let him walk away.  Your child-to-be deserves the father's financial support, so that you don't have to carry the whole load by yourself -- and if this is your first child, you need to understand that this is very serious stuff, because raising a child is expensive.  That doesn't mean you can't get full custody, along with child support.

In fact, since (I'm assuming) you're not married, the law gives you full legal custody of the child at birth, as the mother, even if you put the father's name on the birth certificate.  If you get married, later, on, and your husband wants to adopt your child, that's the time to terminate the father's parental rights, and his child support obligation.