The earliest documented work by the New Haven, Connecticut, architect and bridge engineer Ithiel Town, this small two-story Federal-style house has a primary facade of flush board siding, Ionic pilasters, a balustrade hiding the low-pitched hipped roof, and a doorway framed by sidelights and a lunette. Town's expertise as a designer of timber-framed bridges appears in the unusual
House moving also preserved an eighteenth-century house nearby (20 Gray Gardens West, NRD), a 1798 dwelling moved from Duxbury in 1930. The two-story structure is a vernacular Georgian-style dwelling with a steeply pitched hipped roof, quoins, and a center chimney.
The surrounding neighborhood of Gray Gardens West and Gray Gardens East (both NRD) consists of houses built in a development begun in 1922 and with restrictions allowing only owner-occupied single-family homes. Two well-known women architects, Lois Lilley Howe and Eleanor Manning, built the Colonial Revival house and semi-detached two-car garage at 4 Gray Gardens West in that year. Also erected in 1922, 16 Gray Gardens East challenged the Colonial Revival. There, A. Graham Carey freely used Tudor period motifs, such as floral vergeboard, rope moldings for the casement windows, and gable end ventilation in the form of dovecotes.