I just saw this note. I am a white Irby descended from the Irby family of Panola County. My great-great grandfather, John W Irby, was born in Virginia in 1820. His son, my great-grandfather, James F Irby, was born in Virginia in 1848. The family moved to Panola County some time in the late 1840's, because they show up on the 1850 Panola County Census. If you check the 1860 Census from Panola County, you will see that John W Irby and his wife Maria had a number of children and over 20 slaves. If you look at the slave rolls from the 1850 & 1860 Census, they show 19 & 27 respectively. Unfortunately, the slave rolls did not show the names of the individual slaves, only their age, sex and owner's name. However, if your family lore says you were from Panola County, it is entirely plausible given the documented history of that time. I hope this helps!