Is it legal to charge someone for a trip you invited them on after you get back even though possible payment was not talked about before leaving?

The terms "legal" or "illegal" are essentially criminal terms. What you have described is a civil dispute. If one was invited on a trip by someone and costs were never discussed and such were paid up front by the person who did the inviting, it would seem that under the law that the trip was a gift.