Does a doctor have to issue me a refund me due to incompleted treatment?

First, look at any terms of service or other agreement(s) with the orthodontist which you signed: if that agreement or contract states there are no refunds, there are no refunds--in this case, you contractually agreed that you could not geta refund.
If there was no limitation on refunds, then if they did not complete the treatment, they should owe you some of the money back--a portion equivalent (more or less) to how much treatment they did, so long as YOU did not cancel the treatment or discontinue going. If, say, the improved your teeth mostly but not entirely, you might get between 1/4 and 1/3 back (as an estimate; this is obviously subjective); if there were supposed to be 10 treatment sessions, but treatment was discontinued after 8, you might get 20% back. Etc. They are entitled to be paid for the work they did and/or benefit they provided, even if the treatment was not entirely successful. Of course, even if you "should" get something back, if they won't pay, you'd have to sue, and whether it is worth the time, cost, and effort of suing is something you'd have consider carefully.
If you discontinued treatment while they were still going ahead with it, even if it was because you were dissatisfied, you are not entitled to anything back--you do not have the right to call off ongoing treatment, when they are ready, willing, and able to do their part, and get a refund.