He did not get along very well with the Covenanters and his supporters in England regretted his abandonment of the Church of England. Losing the Battle of Worcester when attempting an invasion of England to take the throne on September 3, 1651, he--like his great-great nephew Bonnie Prince Charlie--had to hide and wear a disguise to escape England. He famously hid in the "Royal Oak" in Boscobel Wood, and Father John Huddleston and he escaped discovery at Moseley Old Hall, hiding there in a recusant-era priest hole!
He and his family would live in exile in France until the Restoration of 1660. That Restoration would be remembered every May for many years on Royal Oak Day or Oak Apple Day, while Charles II rewarded Father Huddleston and the others, especially the Penderell brothers of Whiteladies, once he became King of England, Ireland and Scotland. And of course, Father John Huddleston, protected by the King during the Popish Plot, received Charles into the Catholic Church in 1685 (February 6 Old Style).