On the slope of Fisher Hill overlooking Brookline Reservoir (BR30) stands Brookline's finest example of Georgian architecture. Dr. Zabdiel Boylston acquired the property from his brother Peter in 1736, including the seventeenth-century ell of the present house. Dr. Boylston achieved fame for inoculations against smallpox, despite objections that he was challenging the will of God. In 1766 William Hyslop acquired the sixty-seven-acre estate. In the early nineteenth century, the property passed to Henry Lee, who occupied it principally during the summer season.
Fluted piers supporting a full entablature and pediment frame the principal entrance; the interior holds richly ornamented Georgian woodwork. Between 1870 and 1880, architect John Hubbard Sturgis, known for his measured drawings of the Hancock House on Beacon Hill before its 1863 demolition, rented the house and made a number of