James Madison was the Son of James Madison & Nelly Conway. He was born on the 5th of March 1751. (OS) at Port Conway on the Rappahannock river where she was at the time on a visit to her mother residing there.

His father was the Son of Ambrose Madison & Frances Taylor. His Mother was the daughter of Francis Conway & Rebecca Catlett.

His paternal grandfather was the son of John Madison and Isabella Minor Todd. His paternal grandmother was the daughter of James Taylor and Martha Thompson.

His maternal grandfather was the Son of Edwin Conway & Elizabeth Thornton. His maternal grandmother the daughter of John Catlett & Gaines.

His father was a planter, and dwelt on the Estate now called Montpellier where he died Feby. 27. 1801. in the 78th year of his age. His mother died at the same place in 1829. Feby 11th. in the 98th of her age.

His grandfathers were also planters. It appears that his Ancestors on both Sides, were not among the most wealthy of the Country, but in independent & comfortable circumstances.