Hi Carol My thinking wwas a little more complicated. The records for the women give very little information. In the case of Elizabeth just that she was sentenced in Exeter for stealing bed linen from her master 7 years? first offence Single one son. It would to be expected that if James was with her he would have been mentioned either on her personal record or certainly on the shipping list But he doesn't appear on either, assumption must be that she left her baby in England.
As you say James witnessed Elizabeth's marriage he was certainly in Tasmania at that time
Seems likely that a young man without parents could have got into trouble with the law and followed (it happened).
The records for the men are much more detailed - description / family members etc. If one of the three is her son there will surely be a clue somewhere in the records (native place etc. etc.)
Cheers Denis