Hi,
Thank you for your post, I appreciate it, The marriage cert (16 Oct 1860, to Anne Steer, Plymouth) states his father is called James, Labourer, Frederick states his age to be 30yrs, which also fits in?

On checking the 1871 census I believe should read Syresham, Northamptonshire as there isn't an Eversham in Northants?

It could be that the entry in the 1861 census that you mention is this person
name: Frederic Atkins
gender: Male
baptism/christening date: 28 Feb 1830
baptism/christening place: Stony Stratford, Buckingham, England
being that the Stafford could be Stratford? and the fathers name on the baptism is James as stated on the marriage cert. Frederick is aged 27 on this census which would be out by a couple of years?