"free person of color" was not an incorrect reference used by the government, "person of color" doesnt just refer to blacks it applies to anyone who is not white, so Indians, asians, or any other race that was not pure caucasion would be correctly described as a "person of color" as well as blacks.