IfIi get injured at work and go to the urgent care on my lunch hour but find out that I cannot return to work that day, do I have to file a workman's comp claim?

You need to file a worker's comp claim if you want worker's comp to pay for your treatment of your injury and any time you missed from work.  Because you were injured on the job, your recovery for that injury is limited to worker's comp, and you can not sue your employer.  However, if you do not want to be reimbursed for treatment of your injury and for lost time from work, you do not need to file a claim.