Indeed I would point out that the marriage contract was between the husband and wife. It was the marriage contract that was broken and it was the wife who broke it. The employee had no contract with the husband regarding who he had sex with. Assuming the employee was unmarried he broke no contracts with anyone.

Would the employee be fired for adultery with anyone else? Would other employees be fired for it? Ultimately the employee is guilty of moral turpitude, and the employer should apply the same rules on everyone regarding it.