Encroachment

First step is check the surveys--your and theirs. Where does the line run? You don't need to take their work for it.

If it turns out that your fence is on their land, you may have to move it, though given how long the fence has been up there, it's not impossible that you might be able to claim some sort of right in it (possibly adverse possession, though doubtful, unless you knew it was the other property's land and deliberately built it there anyway; possibly some sort of easement). It would be worth consulting with a  local RE attorney, but definitely check the surveys first.