The Mercer County Historical Society owns three adjacent buildings southeast of the courthouse. The Dr. James Magoffin Jr. house was built in 1821 and occupied by his descendants until 1951, when it was donated to the society. It is a five-bay frame house with double-sash twelve-over-twelve windows, original shutters, and a central double wooden door with flattened arch transom. The gable-roofed structure has an intersecting gabled ell and two shed-roofed additions on the east elevation.
North of the house is the small, frame Episcopal church of 1884 (formerly Church of Saint Edmund the Martyr and now H. B. Miller Memorial Chapel), which was moved to this site for weddings and special services. It has a shingled gable end and a cupola. Behind the house, the long, one-story McClain Print Shop (c. 1830) was moved to this site in 1973 from the town square. The frame Greek Revival building, with pilaster strips and pedimented eaves at the gable ends, operated as a print shop until 1965.