Norfolk's strategic location near the mouth of the Chesapeake Bay situated it ideally for regional and national commerce but at the same time made it especially vulnerable to attack. Sometime during the eighteenth century two earthworks, Fort Norfolk and Fort Nelson, were constructed along the Elizabeth River's north and south banks, respectively. Although well situated, the two forts were severely damaged during the Revolutionary War; their reconstruction under the direction of John Jacob Ulrich Rivardi did not commence until 1794. A view of Norfolk and Portsmouth drawn by Benjamin Henry Latrobe around 1796 depicts Fort Norfolk as having a polygonal berm. Unfortunately for the city's defense, Latrobe's 1798 proposals to redesign Forts Norfolk and Nelson seem to have been scuttled for political reasons. In 1807 the Leopard, a British frigate, attacked the American vessel Chesapeake near Cape Henry. Many of the latter's crew members were killed or wounded and several were captured and impressed. The incident rattled the citizenry and led to the reinforcement of Fort Norfolk. The result must have proved insufficient, since the fort was almost completely rebuilt around 1810, perhaps incorporating portions of the earlier earthwork. Fort Nelson was destroyed to make way for the U.S. Naval Hospital in Portsmouth during the 1820s.
The present appearance of Fort Norfolk is the result of two major building campaigns. The irregularly shaped perimeter walls and earthen ramparts, the main gate, and three major interior buildings can be assigned to the first campaign, which was completed just before the War of 1812. Except for a missing section of hornwork that would have protected an exposed building on the east side of the fort, the brick walls survive in excellent condition; an arcing section facing the harbor is particularly noteworthy. Visitors pass through the original wooden gates and beneath a guardhouse. A vaulted chamber, immediately adjacent to the gate at the right, is popularly known as the dungeon but was more likely a storeroom. To the left is a one-and-one-half-story building formerly used as a barracks. A parade ground extends northward from the gate, and the east side is lined with two Federal buildings: a small storehouse that has been restored as the headquarters of the Norfolk Historical Society, and a larger officers' quarters. Ironically, the reconstructed fort was never tested by military action. A hastily constructed fort downriver at Craney Island repelled advancing British forces in 1813, and the erection of Fort Monroe and Fort Wool at Hampton Roads following the Treaty of Ghent (1814) rendered Fort Norfolk redundant.
The second building campaign at Fort Norfolk dates to around 1850–1855, when the U.S. Navy assumed control of the property from the U.S. Army and converted it into a navy magazine, demolishing some older buildings in the process. A large magazine was constructed on the west side of the parade ground with numerous fireproofing safeguards, including brick walls with granite trim, interior vaults of brick, and copper-lined wooden shutters and doors. A wooden canopy with delicate cast iron supports links the magazine to the officers' quarters, which was extensively remodeled into a facility for the assembly and storage of ordnance. This building contains one of the fort's most unusual attractions. Around 1864, following a tumultuous wartime interlude when control of the fort shifted from the Union to the Confederacy and back again to the Union, a group of captive blockade runners scrawled humorous and somewhat obscene graffiti in a second-floor chamber to while away their time. After the Civil War, concern about safety led to the removal of the naval magazine to a location well beyond the