Hi
Since you have all the basic information from the TFL and IGI marriages, suggest you might find the sort of personal details you are looking for in the newspapers of that time - NLA site TROVE. Could the James Ballard seaman who left Hobart for Sydney in 1822 have been James's (1858) father?