The portrait was commissioned shortly after Lady Margaret’s death, around 1510, by John Fisher, Bishop of Rochester and Lady Margaret’s advisor.
In 1534 he fell out of favour with King Henry VIII, Lady Margaret’s grandson, and his home was raided by the king’s henchmen, who stole or destroyed many of his possessions, including books he had promised to St John’s College Library.
But the portrait of Lady Margaret was safe at the Bishop of Rochester’s palace in Lambeth Marsh and was transported to St John’s shortly afterwards to ensure it would not be destroyed.
By "he fell out of favour" the writer means that John Fisher was stripped of his title as bishop by the new Supreme Head and Governor of the Church of England and imprisoned in the Tower of London.
Thomas More wrote two letters in English to another occupant of the Tower who had refused the Oath as well: Dr. Nicholas Wilson. He had been in the Tower for ten days before More arrived. According to the Dictionary of National Biography, Wilson was a:
Roman catholic (sic) divine, born near Beverley, was educated at Christ's College, Cambridge, graduating B.A. in 1508–9, and commencing D.D. in 1533. He was related to John Wilson, prior of Mount Grace in Yorkshire (Letters and Papers of Henry VIII, xiv. ii. 748). Before 1527 he was appointed chaplain and confessor to Henry VIII (ib. iv. 2641). On 7 Oct. 1528 he was collated archdeacon of Oxford, and in the same year received from the king the vicarage of Thaxted in Essex (ib. iv. 4476, 4521, 4546). Wilson was a friend of Sir Thomas More and of John Fisher, bishop of Rochester, and was a zealous Roman catholic (sic), frequently acting as an examiner of heretics (Foxe, Actes and Monuments, ed. Townsend, iv. 680, 703, 704). On 28 March 1531 he was presented by the king to the church of St. Thomas the Apostle in London (Letters and Papers, v. 166), and in 1533 he was elected master of Michaelhouse at Cambridge. In the latter year, however, when the divorce of Catherine of Aragon was debated in convocation, he joined the minority in asserting that the pope had power to grant a dispensation in case of marriage with a deceased brother's widow. About that time he was employed by the papal party as an itinerant preacher in Yorkshire, Lancashire, and Cheshire. He also visited Bristol, where he encountered Latimer, and threatened him with burning unless he mended his ways (Strype, Eccles. Mem. 1822, I. i. 245; Letters and Papers of Henry VIII, vi. 247, 411, 433, xii. ii. 952). His opposition to the king soon involved him in peril, and on 10 April 1534, a week before the arrest of Fisher and More, he was committed to the Tower for refusing to take the oath relative to the succession to the crown (ib. vii. 483, 502, 575, viii. 666, 1001; Foxe, v. 68). He was attainted of misprision of treason by act of parliament, deprived of all his preferments, and condemned to perpetual imprisonment. Confinement soon caused his resolution to falter. Before his own execution More wrote him two kindly letters, telling him that he heard that he was going to take the oath, and that he for his own part should never counsel any man to do otherwise (More, English Works, i. 443). Wilson, however, hesitated for many months longer, and on 17 Feb. 1535–6 Eustace Chapuys, the imperial ambassador, wrote to Granvelle that it was reported that Henry intended putting him to death (Letters and Papers, x. 308). In 1537 he took the oath, and on 29 May he received a pardon (ib. xii. i. 1315, 1330, ii. 181).
So he remained in the Tower for almost two years after Fisher and More were tried, condemned, and executed! He probably wasn't considered as important as Fisher and More for prosecution and execution.
In the letters (numbers 13 and 14 in this book) More sent Wilson (I don't know if the letters Wilson sent More survived), Wilson seems to have expressed first his decision to take the Oath and then some uncertainty about whether or not he should, perhaps in response to More's reply to his first letter. More's letters are "kindly" because he does not condemn Wilson's decision or even try to dissuade him; he refuses to disclose why he won't take the Oath, yet he recounts some of the background to his decision: how King Henry VIII knew his position on the king's marital matters for a long time; how More had studied the Fathers and Councils of the Church, and how Wilson knew much of this, etc.
If you read the rest of Wilson's biography, you'll see that he just stayed out of trouble until his death in 1548. He was suspected of communicating with recusants and with Rome; he submitted to the authority of the King and Parliament in religious matters but he ended up in the Tower again in 1540 but was released again in 1541. I suppose we could say he survived, but he still endured questioning, arrest, and royal disfavor until he died.