The one-story jail is one of the last buildings to be associated with Thomas Jefferson. It was built according to a plan Jefferson sent in 1823 to his friend Joseph Carrington Cabell. Cabell had been influential in passing the General Assembly Act of 1800 that established Virginia's penitentiary system. Hearing that Jefferson had recently prepared a jail plan for Cumberland County, Cabell asked for his help. Presumably, the plan Jefferson sent to Cabell was the one drawn for Cumberland. In 1823 Cumberland County had contracted with one of Jefferson's builders, Dabney Cosby, to build a jail (CB3), but it was built according to a different plan. The Nelson County Jail constructed by Phillips and Crawford measures approximately 30 × 40 feet. Now painted white, the building is of brick laid in Flemish bond on the front and west walls. Beneath a wide, low-pitched gable roof, a sawtooth brick cornice caps the building's sides. Exterior openings have been modified, but the interior plan retains its narrow, almost dungeon-like, central passage, three cells on each side that now serve as offices, and a small “solitary” cell at the end of the passage that has been converted to a restroom. The cells, separated by unusually thick brick walls, have offset doors, apparently to minimize fraternization between prisoners, and were each originally furnished with a stove and a privy. The three cells on the left were for white male debtors, white male criminals, and white female criminals. Those on the right were for white female debtors, black males, and black females.
All the cells were the same size and design, a curious arrangement for a time when class divisions were strong and both racial and gender discriminations were carefully drawn. According to Cabell, they planned to fit up the old jail with a kitchen and three small rooms for the jailor's lodging and to build a wall ten feet high around the “establishment” with three separate courts, one for debtors, one for male criminals and slaves, and one for females. Care was taken in the construction of what Cabell intended as a model jail in conformance with the era's notions of humane and well-managed treatment of prisoners as outlined in John Howard's Prisons and Lazarettos (1787).