Can a father refuse to let a mother take her children if she leaves him?

If you have any child support and/or child custody agreements in place you want to review them for specific provisions of such matters (kidnapping, leaving the country or the like).  If you don't, typically there is no issue with you leaving, especially if you obtain a restraining order against him concerning the abuse.   If you have joint custody, there could be an issue with the idea of kidnapping or interference with his parental rights based on Louisiana's laws regarding the same.

Every state's law is different in terms of what would be considered kidnapping. In Louisiana, under Section 45, among other definitions of kidnapping is the following: The intentional taking, enticing, or decoying away and removing from the state, by any parent of his or her child, from the custody of any person to whom custody has been awarded by any court of competent jurisdiction of any state without the consent of the legal custodian, with intent to defeat the jurisdiction of the said court over the custody of the child.

It appears your situation would be fine but if there are indeed any orders in place for the benefit of the child, you may wish to think about amending the orders through a court hearing. Again, if you are being abused, you should seek a temporary restraining order which could turn into a permanent one for the sake of your safety and that of your children.