Is this illegal to record someone without them knowing?

The majority of states are what is known as a "1-party consent", including ME. So if the person recording a conversation is a party to it, then the recording is legal. Conversely, if the person recording is not a party to the conversation, then such a recording would be illegal. However, the law does make an exception for in-person communications in which the parties do not have a "reasonable expectation of privacy". So for example, it would be legal to record a conversation in a public place where it might reasonably be overheard even if you were not a party to the communication.

Bottom line, since the comment were directed at you, you were obviously present during the communication. So your recording was legal. However, I would urge that you think before attempting any of this. Your employer might take a dim view of your having made such a recording and it could potentially result in your dismissal.