In 1743, Robert “King” Hooper sold land at what is now 185 Washington Street in Marblehead to Justice William Lee, who had plans to build a new house on the site which were not carried out. Instead, the smaller home located on the property was inherited by his grandson, Col. William R. Lee, a merchant and officer in the Revolutionary War, who hired Charles Bulfinch to design an expansion of his home into a mansion that would resemble that of his uncle, Jeremiah Lee. The house features a wood exterior designed to resemble granite.