Your Joseph E Mathieu is probably related to my Mathieu side from Chicago. As a matter of fact their is a Foseph E Mathieu in my family born 1870. his older brother is Exor joseph Mathieu, born to Louis Mathieu and Mary (Breault) 1863 Chicago directory has a Louis living with two joseph Mathieu at 25 Hope st. and their occupations are mitten makers. Louis was born in Canada in 1845 and I'm thinking that the two Josephs are his brother and the father (both named Joseph). Possible that either of these Jeseph's could be the father of the joseph you seek in 1862. All my mathieus are buried in mt. Carmel or St. Joseph's about a quarter mile away.
Just sent away foe louis mathieu death cert. hope it wikk have his parents. Also, look towards Kankakee,Illinois valley. French Canadians flooded into the valley 1830/40's and I see Mathieu's there.