As originally established by the House of Burgesses from Northumberland County in 1653, the territory of Westmoreland County encompassed much of what later became the various counties and cities of Northern Virginia, including the city of Alexandria, Arlington County, Fairfax County, and Prince William County.

These were part of Westmoreland until 1664, when Stafford County was formed. Westmoreland County was the birthplace of George Washington, the first President of the United States (at the former settlement of Bridges Creek, Virginia);[3] of James Monroe, the fifth President; and of General Robert E. Lee, commander of the Confederate armies. 

The county was the place of residence for Colonel Nicholas Spencer (1633-1689), who patented the land at Mount Vernon in 1674 with his friend Lt. Col. John Washington, ancestor of George Washington. Spencer, who served as President of the Council and acting Governor of the Colony of Virginia, was the cousin of and agent for the Barons Colepeper, proprietors of the Northern Neck. He lived at his plantation Nomini, which his descendants later sold to Robert Carter I. Robert Carter's grandson, Robert Carter III, voluntarily freed almost 500 slaves from Nomini Hall beginning in 1791 and settled many on lands he gave them. His manumission is the largest known release of slaves in North America prior to the American Civil War and the largest number ever manumitted by an individual in the U.S.[4