The Walter Baker Chocolate Company's many surviving buildings occupy a stretch of the Neponset River, which has fostered industrial enterprises since Israel Stoughton built his gristmill here in 1634. The rapids powered an array of industrial enterprises, including James Baker's 1765 sawmill annex for grinding cocoa beans. Over the next ninety years, his son Edmund and grandson Walter built a thriving chocolate manufacturing concern. Walter Baker's nephew Henry Pierce, who later became mayor of Boston, headed the company in the 1850s and embarked on a program of aggressive expansion, including the complex's most significant buildings. Major construction continued under the group of local capitalists who succeeded Pierce, until the General Foods Corporation acquired the company in 1927; they moved the facility's operations out of state in 1965. The various Walter Baker Chocolate Company's buildings are held by a number of owners and are largely occupied by condominiums, warehouses, small businesses, and light manufacturing.
Most of the Baker Chocolate Factory's buildings were designed by prominent Boston architect Nathaniel J. Bradlee and three later partners—Walter Winslow, George Wetherell, and Henry Bigelow—whose monumental, finely detailed red brick structures help unify the tightly knit complex centered around Adams Street and the Neponset River. Particularly notable is the 1872 three-story Second Empire Pierce Mill at 1220 Adams Street, with its four-story projecting central and end pavilions capped by peaked roofs and cast-iron cresting. Behind stands the five-story brick Steam Mill, built in 1868. The site's two largest mills, the 1891 Baker Mill at 1245 Adams Street and the adjacent 1911 Forbes Mill to the north, share the same Romanesque Revival detailing and are joined across Baker Court by an upper-floor copper-sheathed connector. The Georgian Revival administration building (1918–1919, George Shepard), presides over the complex up the hill at 1231 Adams Street.