In the twelfth century the Fitzpaines were an enterprising family who acquired great landholdings in the South West, particularly in Devon, Somerset and Dorset. Some places still carry the family name, such as Cheriton Fitzpaine. The family held the Manor of Witheridge and Robert Fitzpaine was granted a charter to hold a midsummer fair for the village in 1248. This Robert was the son of Roger, who died shortly before 1237. There followed a succession of Roberts and it is difficult to distinguish their individual dates, but in 1299 the family was raised to the peerage and became the Lords Fitzpaine.

Shortly after this they acquired the lands and properties, which had belonged to the de Courci family of Stoke Courci (now Stogursey) in Somerset. This land included the nearby manor of Cannington, where, in 1138, Robert de Courci had founded a founded a nunnery for Benedictine nuns, and endowed it with 3 acres of arable land and 3 acres of meadow. The priory was very influential in the life of Cannington, although records show that by the 14th century there was a degree of corruption and various inquiries revealed misuse of funds and illicit wanderings by the nuns. The Nunnery was suppressed in 1536 during Henry VIII's imposed dissolution and new buildings, incorporating parts of the priory, were erected. The "foundership" of a religious house became part of the estate and became the "property" of either his heirs or future purchasers of or grantees. In this way, the Fitzpaines became founders of the Convent of the Blessed Virgin Mary, to give it its full title, although it is often referred to in documents of the period as Cannington Priory.

A Robert, probably grandson of the first Robert, is mentioned in several documents connected with the first convent, in particular with reference to matters concerning the election, appointment and installation of a prioress from 1317 to 1336. Election procedures were very strictly laid down - in a 'secret ballot', with each member of the community giving a verbal vote privately to the adjudicator. Two elections were held in 1317 because the legality of the first procedure was contested.

In 1326 Robert sought, and was granted, the Kings permission to purchase 24 acres of land in Cannington and Radweye. This gift was to support a chaplain to celebrate a mass every day for the good of his souls, the souls of his ancestors and inheritors, and all the faithful departed. It added to the wealth of the priory, but not to that of the nuns who were responsible for ensuring that the terms of the gift were faithfully executed. The community like many others, had considerable wealth for which they were responsible, but much of it was for specific purposes and could never be used for 'living expenses'. The nuns made vows of poverty, chastity and obedience; indeed, they had little option but to keep the first. Sometimes they received small personal bequests, but very often it is recorded that they were excused paying taxes because they were so poor.

The Lords Fitzpaine did not survive for very long, the last, Robert (II). Lord Fitzpain, dying in 1354 leaving no male heir. His only daughter Isabel was his heiress, and she took all the lands to her marriage with Sir John de Chidiok who became the new founder of the convent. It is said that during the period immediately before Roberts death the Black Death had raged through the country causing the demise of around a quarter of the population. Everywhere was in disarray and even at the convent discipline broke down. Robert had been required to help restore order and the Bishop attended a hearing when the disobedient nuns were disciplined. The prioress herself was also held to account for granting Corrodies. This was a kind of "get rich quick" insurance policy whereby for a one off payment, a non-religious woman would be allowed to live with the community for life. The mere fact that such events took place indicates the extent of desperation felt by these women. Not only was it against the law, but it also reveals the level of chaos and shortage of food caused by the effects of the Black Death, and also the abnormal rainfall in the summer of 1348. It was shortly after this that Isabel, through her husband, Sir John de Chidiok, made a gift of 12d in rents in Witheridge. The priory was also granted the advowson of the church, which allowed it to present suitable prospective incumbents to the bishop. The advowson also brought with it the obligations to defend the rights of the Church. This gift must have come as a great relief as well as being a significant boost to morale.

The first rector recorded as being presented by the priory is Robert Crosse. He had only been in the village for a year when he asked if he might change with Henry de Lutton, the vicar of Spaxton (near Bridgewater). He described the parish as 'large and widespread' and said that he found the 'cure of Witheridge great and onerous', which he could not rule according to his conscience because of the diverse occupations he had with the Bishop of Bath and Wells. Henry de Luttons reason for accepting the post was not so much from a desire to move to a 'large and widespread' parish, as to get away from his next-door neighbour. He said he was in dispute with Lord James de Audelegh over a land matter and he 'dare not pursue the case while in the neighbourhood by reason of his (Audeleghs) deadly hatred' for him. Ralph of Shrewsbury, Bishop of Bath and Wells, agreed to the exchange and the matter was duly recorded in the Bishops register, with the exchange taking place during the Feast of the Conversion of St Paul, on 25 January 1362. Both vicars being instituted into their new parishes on the same day. Parson Henry de Lutton remained at Witheridge until 1379 when he exchanges again, this time with Edmund Demalmeshulle, rector of Stormborough, Kent. There followed a succession of rectors of whom we know little except for their names.

Cannington lay close to the flood plains of the River Parrett, and in 1427 disaster struck when high tides and flood lands engulfed most of the priory lands. Consequently the Pope was petitioned for more and in 1428 lands were granted from Witheridge. The fact that the Pope and Bishop were concerned with this grant would tend to indicate that the lands were already church property. This part of the village became Witheridge-Cannington, the rest being Witheridge-Bradford, and, to ensure that the vicar received sufficient funds from the land to fulfil his obligations to the Church, and to support himself, the Bishop set out in great detail which land belonged to the nunnery, and which to the vicar. This document tells us something of the agriculture of the time. The vicar received the tithes of beans, peas, and apples growing in the ancient gardens, the tithes of coppice wood and that of calves, colts, lambs, suckling pigs, kids, geese, pigeons, eggs, wool, milk, cheese, butter, plus flax and hemp growing in the ancient gardens. The document also describes the boundaries of the lands, but without a map, it is difficult to understand. We do know, however, that he was to have 'the dovecote within the garden' and a 'certain meadow of one acre at the upper end of which a certain fish pond is situated'. The house in which the vicar lived was known as the manse; the vicarage being not just a building, but rather all the lands, gardens, and tithes which were assigned to him to enable him to support himself and carry out various church obligations. The document endowing the nunnery with the Witheridge lands and setting out the conditions was signed and sealed by the prioress, Johanna Childeldon, in her Chapter House at the priory on 16 December 1428 and by the bishop three days later.

The Guide books for the churches at Witheridge and Cannington suggest that both buildings were rebuilt in the fifteenth century. On the outside, they look very different with Witheridge being built from the grey bedrock stone that befits a rugged upland parish, and Cannington of local red sandstone. The tower on the present church was not and original feature being added in 1574, and if there was one with the original building, it was not retained. At Cannington, the new building was added to the existing tower. Inside the two churches there are similarities. Each has two arcades of arches, five on each side at Cannington and four at Witheridge. The Cannington arches are taller and more slender, but the carving on the capitals does not bear comparison with the fine and beautiful work at Witheridge. Each church has a single spanned roof that covers both nave and aisle, and all ceilings contain many carved wooden bosses.

By 1440 the priory had a new founder, Robert Poynings, and as his family crest appears on some of the new work at Cannington, he must have been responsible for some of the rebuilding along with the prioress. She would also have been informed about progress at Witheridge through the vicar and her steward. Although the priory had an interest in the building of both churches, it is unlikely that any of the nuns would have seen Witheridge. They were enclosed in their own part of the monastery and needed the Bishops permission to leave the outer wall that surrounded it. The last appointed prioress was Cecilia Verney in 1504,I and she remained until the dissolution of the monasteries in 1536. In 1521, she presented her brother George to the living at Witheridge, a position he held until his death in 1562. It appears that members of the community at the time of the dissolution were given the option of leaving penniless or moving to a larger surviving house, Prioresses being the only ones to receive a pension at this stage. Records show that some of the nuns moved to Shaftsbury Abbey, whilst two moved to Polsloe Priory in Exeter.

The last link with the priory was the vicar, George Verney. He remained vicar at Witheridge throughout the reigns of Henry VIII, Catholic Mary, and Protestant Edward VI, but died four years into the reign of Good Queen Bess. He died in 1562, after 41 years in office unmarried but highly respected. He must have given the parish a sense of security in very changing times as well as continuity which carried it through from the end of the medieval period into a new age.