The Somerset Archaeological and Natural History Society was founded in 1849 following discussions among "several gentlemen of Taunton and its neighbourhood." It was one of several such organisations established in the English shires during the 1840s, and reflected the early Victorian flowering of interest in county history and archaeology, as well as in the natural environment. By 1851, the Society had 420 members - including large numbers of the Somerset gentry and clergy - and published that year the first volume of its annual Proceedings. It was also beginning in those early years to collect items for a society library and museum.
The Society's early home was in the Victoria Rooms (since demolished) at the centre of Taunton. In June 1874, however, the status and ambitions of the Society were dramatically transformed when it acquired Taunton's medieval castle as its new headquarters. The castle had for centuries been the administrative centre of the great manor of Taunton Deane, owned by the bishops of Winchester, and achieved notoriety in 1685 as the setting for the Bloody Assizes. In spite of work undertaken in the late 18th century to convert part of the building to judges' lodgings, by 1874 repairs were urgently needed. In 1884 the Somerset Room (the Great Chamber of the medieval castle) was reroofed. In 1899-1900 the Great Hall was repaired and refitted as the Society's chief museum space, and in 1908-9 the Adam Library was created to house the Society's growing collection of books. By the early 20th century the society had not only succeeded in rescuing Taunton Castle, but had secured a national reputation as a collector and publisher, and as a promoter of field meetings and archaeological excavation.
No individual achieved more for the Society than the able and combative Harold St George Gray, Assistant Secretary, Curator and Librarian from 1901 to 1949. He began his career as assistant to General Pitt-Rivers, and from 1904 collaborated with Dr Arthur Bulleid in excavating the Iron Age lake villages at Glastonbury and Meare. Gray directed other excavations throughout the county, and as an administrator oversaw the building of the castle's Wyndham Gallery in 1934. It was Gray, as well, who guided the Society through its greatest controversy when, in 1922, Frederick Bligh Bond was dismissed as Director of Excavations at Glastonbury Abbey. Bond had relied increasingly on help he claimed to have received from the spirits of Glastonbury monks.
The museum was given its present basic form by Wilfred Seaby, Gray's successor, and by the time of Seaby's departure in 1952 plans had been drawn up for the installation in the Great Hall of the 18th century staircase from St Mary Redcliffe Vicarage, and negotiations were complete which would bring the Low Ham Roman pavement to Taunton. The financial difficulties which had beset the Society at least since the Second World War were greatly eased in 1958 when the castle and the museum collections were leased to Somerset County Council for 49 years. Staff of the County Museums Service remain the day-to-day managers of the castle and its collections, though the Society has retained use of some space in the estate. The Society's library was transferred in 2010 to the Somerset Heritage Centre adjoining the County Council's Somerset Studies Library. There, the two collections now form one of the finest local studies libraries anywhere in England.