Should I punish my son or ignore his rude comment?

Your both at fault. I would be mean to my mom if she called me mean things. you should be the bigger person. dont punish, apologize! then maybe he will stop if not then you can punish him without calling him mean things because you are mentally scaring your child for life! those names hang with them until an apology is made but sometimes even longer