Owner Horatio Gambrill built the first section of Druid Mill in 1866, a long two-story stone building with an Italianate tower. Another one-story building added in 1872 doubled the size of the operation, with weaving located in the newer structure. It used coal-generated steam power from the very beginning, leading the post-Civil War change among the Jones Falls mills.
In 1892 the complex became Mount Vernon Mill No. 4, before being added to the Mount Vernon Woodberry Cotton Duck Company consolidation in 1899. In 1917 Poole and Hunt purchased the property for manufacture of washing machines and other appliances. After a variety of owners between 1935 and 2009, Hamel Builders converted the nineteenth-century mill buildings and boiler plant, now renamed Union Mill, into offices for nonprofits, low-cost housing for teachers and city employees, and small shops.