This house, now altered, originally resembled the Willoughby-Baylor House: a two-and-onehalf-story brick Federal town house. It was probably built by Matthew Hervey, a Norfolk merchant who, in 1802, sold the house to Harrison Allmand, another well-to-do merchant. The plan is the standard side hall with double parlors. Sometime around 1820 the house received a Greek Revival facelift. The main facade was stuccoed and quoined, and stone lintels with Latrobe-inspired ornamented corner
Allmand's granddaughter married into the Archer family, and the house was occupied by her descendants until 1973. At present a fraternal organization uses the house for its offices.