As you say - the will of Joseph James SWABY sets out a fair record of his family. The will dated 1807 was subject to probate in 1812 so puts paid to the errors repeated by many that he died in Yorkshire in 1821 - his son aged 32 did.
I have come at this from the direction of the PEART line through an extensive list of marriages so I have no real connection - it was Frederick PEART who married Ann Margaret Eleanor SWABY 5 Mar 1842 at the Hope. They had 4 daughters who all appear in England from 1851, I can find no son which has been mentioned by others. She then married to William Henry TYLER sometime about 1850/51 - 1s & 4d resulted - all found in England from 1871. He died before 14 Jul 1858, the baptism of his youngest daughter; She re-marries again about 1859 to William WARD and, as far as I can determine has 1s who is found in England from 1881.
All I have found including newspaper summaries are on my tree.