Here's what the 1880 Anderson Co. census has on your Samuel Brown and family:
1880 Census Place: Palmer, Anderson, Kentucky
Source: FHL Film 1254401 National Archives Film T9-0401 Page 320B
Relation Sex Marr Race Age Birthplace
Samuel BROWN Self M M W 46 KY
Occ: Farmer Fa: VA Mo: VA
Nancy J. BROWN Wife F M W 46 KY
Occ: Keeping House Fa: VA Mo: KY
John B. BROWN Son M S W 15 KY
Occ: Works On Farm Fa: KY Mo: KY
Abraham BROWN Son M S W 11 KY
Fa: KY Mo: KY
Julia BROWN Dau F S W 10 KY
Fa: KY Mo: KY
Elmira BROWN Dau F S W 9 KY
Fa: KY Mo: KY
Edword BROWN Son M S W 7 KY
Fa: KY Mo: KY
Sarah F. BROWN Dau F S W 5 KY
Fa: KY Mo: KY
Duran BROWN Son M S W 1 KY
Fa: KY Mo: KY
Hope this helps.