I don't know about the family but did you notice his mother's name is Barfield (spelled Barefield) on the 1880 census? Have you been able to track down any death information for his siblings??