This, Vermont's second Gothic Revival church, was built for the state's oldest Episcopal parish. The church fully reflects high-style Episcopalian design in the northeast. Its Anglican-founded community was eager for a building in the image of an English country church. To this end they hired English builder William Passman of Aulston, Yorkshire, who had been working in Troy, New York. There, the recently completed (1828) St. Paul's Episcopal Church was a near replica of arguably the most significant Gothic Revival design of its day, Ithiel Town's Trinity Church (1817) in New Haven, Connecticut. For Arlington, Passman constructed a church that, despite simplifications, was so close in design to the one in Troy that it suggests that he had a hand in the earlier building as well.
St. James is a simple rectangle with a square tower at its east entrance end. The walls are dark blue local limestone, with dressed pilasters bounding the tower and the front corners of the main block, and dressed surrounds framing the pointed-arched windows and Tudor-arched door. Wooden cornice moldings and a quatrefoil frieze on the tower cap the masonry at the level of the eaves. Flush boards define the pedimental gable and the upper two stages of the tower. The tower is richly embellished with paneled corner pilasters decorated with trefoil heads, elaborate wooden tracery screens on the second stage and framing the louvered openings of the belfry above, and tall, richly cusped wooden finials.
Despite the addition of a chancel and the alteration of the original paneled double-leaf doors in 1898–1899, St. James established and remains an important prototype for Vermont's early Episcopalian high style. Its scheme was refined and repeated for St. Paul's Cathedral in Burlington (1830–1832, Ammi B. Young; burned 1971) and the first Trinity Church in Rutland (1833, John Cain; demolished 1866). Passman was not directly involved in either of these since his compensation from the Arlington parish included fifty dollars for his passage back to England.