Sited at the bend of Main Street, the Athenaeum is a building of many distinctions. This National Historic Landmark was the most expensive private building in Vermont at the time of its construction, one of the state's earliest Second Empire public buildings, one of the first endowed public libraries in the United States, one of the earliest with open stacks, and the oldest unaltered art gallery in the United States. The project's benefactor was industrialist Horace Fairbanks, who wanted to enrich the intellectual and cultural life of his community through the library as an institution and through architecture that embodied the progressive tastes and techniques he encountered during business trips to New York. It is the only extant documented building of New York City architect John Davis Hatch III, best known for Greystone, a Second Empire mansion in Yonkers, New York, that was razed in the 1950s. A founding member of the American Institute of Architects (AIA), Hatch was an advocate of timber framing with masonry infill and of advanced ventilation systems with rooftop monitors, both of which he employed at the Athenaeum. His drawings for the building are preserved in the Athenaeum's collection.
The library was built first, a massive three-by-five-bay timber-framed, brick-veneer structure with a dressed granite base and a steep mansard cap of imbricated slate. It has two principal floors, library below and public hall above, marked respectively by segmental-arched first-floor windows and tall round-headed wall dormers in the mansard roof above the weighty bracketed cornice. A stocky, projecting entrance pavilion is capped by a double mansard, now shorn of its original cast-iron cresting and flag pole. Heavy concentric Romanesque arches with a keystone that helped support a front balcony (removed 1956) framed massive oak-paneled doors (replaced but conserved), giving onto a Minton-tile-lined vestibule with a grand carved staircase. This opens to a suite of well-preserved reading, book, and office spaces with interior detailing by Packard, who oversaw the building's execution. The sixteen-foot-high rooms are filled with floor-to-ceiling bookcases with galleries and spiral staircases in ash and walnut (the upper levels were added in 1882 and 1892), original reading tables, and gas chandeliers that were converted to electricity in 1901.
The art gallery was built between 1872 and 1874 as a wing to the rear of the library. Hatch designed it as a setting for Albert Bierstadt's largest painting, the 10 × 15–foot Domes of Yosemite (1867), acquired by Fairbanks through the architect's brother. It is a cruciform-shaped room with black walnut woodwork and cases, illuminated by a forty-foot-high skylit ceiling on iron and wood trusses and a gas-fueled lighting frame (since electrified) suspended fifteen feet above the floor. Filled salon style with a large collection of late-nineteenth-century American and European sculpture and paintings, assembled by the Fairbanks for this purpose, the interior preserves the experience of a Victorian-era art gallery.