Charles II was a notably unfaithful husband, just like his brother, the Duke of York. (I believe the commonly quoted gossip is that his brother's mistresses were all ugly, however.) Charles II is also known for having both Catholic and Protestant mistresses, although the putative religious observance of the mistresses seems not to matter that much to us now.
At the time, however, it did have an impact--Nell Gwyn once saved herself from a mob by announcing that she was the Protestant whore!
On October 9, 1709, Barbara Palmer, Lady Castlemaine, the First Duchess of Cleveland died. She had been born Barbara Villiers. In 1659 she married Roger Palmer, who sadly became known as Europe's most famous cuckold. They remained married, even though she continued to be the mistress of the Earl of Chesterfield and then became mistress of Charles II soon after the Restoration. Palmer was completely loyal to his Catholic faith and to the Stuart monarchy: the former prevented him from divorcing her, the latter from protesting against her infidelity, since it was with the King. In spite of his high position in the Court, because he countenanced his wife's relationship with Charles, he was arrested during the Popish Plot but defended himself ably enough in Judge Jeffrey's Court.
Barbara herself became a Catholic in 1663 while her children by Charles had the last name "Fitzroy" and he gave them titles as she remained "Lady of the Bedchamber". One of her daughters, Barbara, joined the Priory of St. Nicholas at Pontoise in Normandy in 1691, after bearing a child out of wedlock. After 1673, the king's attentions turned to another "Catholic" mistress: Louise de Kerouaille, the Duchess of Portsmouth.
Roger Palmer died in 1705 and Barbara four years after. She is an important character in Kathleen Winsor's novel Forever Amber, as scandalous as she was in its own day (1944) and was depicted in the Linda Darnell movie (1947) with George Sanders portraying Charles. Natalie Draper played her.