I'm not sure I have anything to help you, but I am responding on the outside chance there might be a connection with the Baldwin I am researching in Brooklyn.

1880 US Census Brooklyn, Kings, NY ED 250

Harriet [Baldwin] McBride 36 Widowed Born VA Fa NY Mo VA
James McBride 19 Laborer Born NY Father Scotland Mother VA
George Wm 7 Son Born NY Father Scotland Mother VA
Mary A Daughter Born NY Father Scotland Mother VA

Harriet (Hattie) married my gr gr grandfather George McBride, was born around 1845 in VA and may have been from Norfolk.