Crossed Rifles on the cap is the US Army insignia for an Infantry Unit. The number 6 would be the 6th Regiment or Company. Now the Crossed Rifles were not used until the mid 1870s, replacing the Horn. So this is not a uniform cap that would have been worn in the Civil War. Where was he living in 1870 and 1880? That would help with identify the state he enlisted in.
But there were photography parlors even back then that had Uniform and other Costumes for a picture.
There were not that many African American army units at the time. Here is a website with the units:
http://www.buffalosoldiers-amwest.org/history.htm