Probably the most notorious accusation against Thomas Jefferson is the persistent allegation that he secretly took a mulatto slave named Sally Hemings (or Hemmings) as a mistress, and fathered several children by her.
The charge was first made in September 1802 (during Jefferson's first term as president) by a Scottish immigrant named James T. Caller, an embittered alcoholic and hypochondriac. Writing in a Richmond newspaper, Caller cited no evidence for his accusation, merely claiming that it was “well known.” On later occasions he changed details about how the affair allegedly began and the number of children supposedly produced by it.
To those who knew Jefferson, his high moral standards and his deep devotion to his dead wife's memory, the entire story was absurd and contemptible. Nevertheless, it soon gained widespread circulation and many believers. Today it is occasionally given credence by black or leftist academics.
Jefferson never replied publicly to the charge. In a letter to a friend in June 1816, he wrote, “I should have fancied myself half guilty had I condescended to put pen to paper in refutation of their falsehoods, or drawn to them respect by any notice from myself.” Years later his grandson, Thomas Jefferson Randolph, maintained that Sally's children were fathered by a nephew of Thomas Jefferson named Peter Carr, thus suggesting an additional reason for his silence.
Serious scholars of Jefferson's life reject the Hemings story. University of Virginia professor Merrill D. Peterson, a prominent Jefferson specialist, commented in his comprehensive biography of the third president: “… It is difficult to imagine him caught up in a miscegenous relationship. Such a mixture of the races, such a ruthless exploitation of the master-slave relationship, revolted his whole being.” (Thomas Jefferson and the New Nation, New York: Oxford University Press, 1970, p. 707.)