Try -familysearch.org- I found three records that show the Joseph Cadby who married Ann Stamps in 12th Dec.1858 at St. Philips, Birmingham - was still married to her in 1881. They are living in Harbourne, Staffs.
The parents of Joseph are, William Cadby and Charlotte Bradney, they married on 27th. August 1822 at St. Philips,Birmingham. I also found a marriage, William Cadby to an Eliza Philips, 10 April 1825 also at St. Philips, Birmingham. It is possible Charlotte died in childbirth and he married again?
I cannot find a record of Joseph to Harriet Alcock but suggest that the above people are the ones you were seeking.
Many regards from a lady with a Cadby family in Wilts.