If my ex-wife has filed 2 complaints in a row against me alleging abuse with CPS, both of which were found meritless, can I sue her for harassment for these false complaints?

You have a couple of different options. 
One is to file a defamation suit for a false allegation.  Many people who are the subject of a false allegation want to pursue this option, but eventually opt out because of the expense and the rekindling effect of emotional turmoil.  Most attorneys will advise you that defamation suits are also very diffcult to prove... which adds to the reluctance of people to file these suits.  This is not to say it's impossible... just difficult.
The second option is to file a suit to modify the custody arrangement.  Your basis would be that the mother has engaged in an intentional pattern of trying to alienate you from the children by filing these false complaints.  If there were periods of visitation that you missed because of these false complaints, then you could also request the court to grant you make up time for the missed visits.
The third option is to visit with an attorney to see if any other details of your case (that you may not have set out in your question), would potentially provide other modification or enforcment options.
Regardless of what you do, make sure that you obtain a copy of the closure letter which identifies the case was closed as "reason not to believe" or some other similar finding.  You need to scan a copy of this letter as well so that you always have a preserved copy.  Sometimes, CPS will delete files, and four or five years down the road, this letter may no longer exist when you need it. If you can get an attorney to help you get  copy of the entire file while it is still recent, that would help as well.  You want it for the same reason... documentation of what did or did not happen.  If you are having problems with the ex now, there is a good chance that the problems will continue down the road, and you want to keep it handy and accessible.  A few years from now, she'll mention that there have been a couple of CPS investigations.  Without documentation, it will be hard for you to prove that the investigations were false.
Additionally, you need to stay on track with being a good dad.  It is a temptation to get the kids involved in the problems.  Keep the kids, at least on your side, out of the middle.  If you ever go back to court, judges tend to frown on those parents that put the children in the middle of the dispute or communication.  Just continue being a good dad and that will make a judge happy... and thus, more likely to rule in your favor during any future litigation.