Found your inquiry most interesting. You found Albert Lewis born South Carolina included in a household in Middletown Ct in 1860 US Census - comparing the members of the household in 1860 with those in the 1850 & 1870 & 1880 households I think that Albert was residing with the family of Joseph Lewis deacon of the South Church of Middletown Ct. Joseph Lewis died in 1858 and his funeral was mentioned in many newspapers in CT. Mary Lewis, widow of Joseph lived to at least age 94 (1882) according to a newspaper article about a birthday celebration attended by multiple generations of her family. The Lewis Family was identified by census enumerators as white; however, that may have been an assumption. I had assisted with work on a family tree whose members were identified differently throughout the decades.
How your great-great grandfather came to live with the Lewis Family in Middletown is not known to me, however, you think that he was in CT prior to the Civil War in a community that had strong abolitionist views & is alleged to have been a stop on the Underground Railroad. You might want to correspond with the South Church (www.southchurchmiddletown.org
) about the Joseph Lewis Family. There are several historical societies in Middletown that might have more information about individual families and their involvement in the abolitionist movement and the Underground Railroad.