It was in Ulverston, Lancashire that he met for the first time Judge Fell and his wife Margaret. Margaret soon became a Quaker (incidentally the name ‘Quaker’ came about because Fox told one of his judges that he should quake at the power of God) but her husband never did although he remained sympathetic to their cause. From now on Swarthmore Hall became the headquarters of the Quakers. The Fells were well respected in the area and they held a lot of influence so they were able to protect Fox and his leaders from a lot of persecution. Swarthmore Hall was a safe haven where the Quakers could have meetings and plan for the future.
Swarthmoor Hall was built around 1586 by George Fell. It was sold by Thomas Fell's grandson in 1759 and in 1912 it was bought by a direct descendant of Thomas and Margaret Fell; however, by this time it was in a dreadful state of repair. It was originally around double the size it is today, half of it being pulled down in 1777. The house had major renovations around 1920 and in the 1960's it was refurnished with furniture from the mid-seventeenth century to make it look as it might have done in Fox's time. It is not known how the house looked at the time and there is little of the original interior left. However, the bed above was given to Fox in the 1670's by Quaker plantation owners in Barbados while he was travelling there. It is a travelling bed and weighs about a ton. The two chairs belonged to Fox (on right) and Margaret Fell.