In 1841 your Charles Alderton was enumerated in Woodbridge Union Workhouse, Nacton, Suffolk, aged 13, next to next to Mary Ann Alderton aged 11, next to Caroline Alderton aged 7, described each as pauper.
In the meantime in 1841 in North Street, Brightelmstone, Sussex, (about 120 miles away from Woodbridge) Charles Alderton of Sussex and his wife Frances Bignall were enumerated as below: They already had a daughter Mary aged 2. Charles was working as a tailor and in the 1851 census he describes himself as house proprietor. I hardly think if he had any other children, and disregarding the duplicate names, that he would have 'left' them in a workhouse in Suffolk.
Charles Alderton abt 1801 Sussex, Tailor
Frances Alderton abt 1806 Sussex
Edwin Alderton abt 1826 Sussex
Alfred Alderton abt 1828 Sussex
Adelaide Alderton abt 1830 Sussex
Agnes Alderton abt 1832 Sussex
Thomas Alderton abt 1835 Sussex
Maria Alderton abt 1837 Sussex
Mary Ann Alderton abt 1839 Sussex
If you have any evidence that your Charles was their son, of course if would be helpful to share it.