A nunnery was established a short distance from the village at Nun Appleton in the 12 century by Adeliza or Alice de St Quintin during the reign of King Stephen, grandson of William the Conqueror.

The Fairfax family acquired the site after the dissolution of the monasteries and built a large house on the site, which became the home of Thomas, 3rd Lord Fairfax of Cameron. Lord Fairfax was famed as a gallant and courageous parliamentarian commander in the English Civil War.

The Fairfax family owned the manor until 1711 when it was sold to the Milner family, who made their money from the Aire Calder Navigation Canal.