If you search the 1850 Census you will locate James A taylor and his then family.

There is a find a grave listing and it provides additional information. It seems that James A had three wives. The First Winny Lambert in Monroe Co Ala the first three sons are from that marriage. Winny died abto 1845. James was in Carroll co by 1850. Emily Caroline Duke was the second wife , He had six more children.