If I get fired can my employer legally refuse to pay me my accrued vacation time?

New Jersey does not have a law generally governing accrued vacation time, so it is up to company policy. The company needs to follow it's established rules or policy for vacation time payouts. If you have an employment agreement that mentions what happens, that controls; if not, but there is a clear written policy in an employee handbook or the like, that will control; if not, then look to what they have done in the past--until they actually change the policy, they need to treat all employees alike.

In answer to your question, if it is in fact their company policy to not pay accrued vacation time when someone is fired, that's what they may do.