Can I be charged with assault in if it's the victims word against mine?

Yes, you can still be charged with assault if there is probable cause to believe that you were the initial aggressor and committed the assault. The fact that the alleged victim has no witnesses, only their testimony of what happened, means that if you are charged and decide to go to trial, the victims credibility will be a key factor in the prosecutors attempt to prove their case beyond a reasonable doubt.