With its austere Doric portico protruding from a massive but simple cubic block, this is the boldest of Jay’s surviving Savannah houses. The abstract classical design confidently uses dramatic lighting effects, inset arched windows, and elliptical interior walls. Particularly striking is the adjoining carriage gate, a triumphal arch flanked by pairs of squat, archaic Doric columns with exaggerated entasis. The side porch on the south features cast-iron columns and a balustrade, while a cast-iron anthemion-like cresting runs across the rear cornice.
Scarbrough was president of the Savannah Steam Ship Company when he built the house and hosted President James Monroe here in 1819 (on the occasion of the launching of the SS Savannah, the first steamship to cross the Atlantic), but he lost everything to bankruptcy in 1820. The grand house was then used as an orphanage, and became a school for African Americans from 1878 to 1962. It has undergone significant modification and restoration, including the 1836–1837 addition and subsequent removal of a third story as part of the extensive 1970s restoration carried out under the direction of the Historic Savannah Foundation, which was headquartered here from 1976 to 1990. Further renovations in 1995, under the ownership of Mills B. Lane IV’s Beehive Foundation, added a new roof and a rear main entrance, transforming the building into a maritime museum. The adjacent 1.25-acre north garden is its most recent addition.