Hi Joe - After a TOTAL rethink... perhaps Tempest Holt may not have been a soldier but a servant who travelled with his employer.

These records seem to be the nearest...

Birth: Dec 1839 Holt Henry Beaminster 8 13 (perhaps his father was a Tempest?)
Birth: Mar 1844 Avery Sophia Axbridge 10 340 (nr Bristol)

Marriage: Sep 1866 HOLT Henry W & EVY (Avery?) Sophia M Bath 5c 985

1871 Census: Sophia Holt 24 Hampshire bc1847

1881 Census:
11 Park Street, Bristol St Augustine, Gloucester (apartments?)
Sophia Holt wife (Head) M 35 F bc1846 Bristol
Harry Holt son scholar 7 M (bc1874) Paris (British Subject)
http://visitbristol.co.uk/things-to-do/park-street-p23531

Perhaps Harry's parents had him registered on their return to the UK.
Births Mar 1876 Holt Henry Chard 5c 464 (nr Teignmouth & still seems nearest).

Deaths Sep 1912 Holt Henry 63 Beaminster 5a 350
Deaths Sep 1918 Holt Sophia 75 Chard 5c 406
Deaths Dec 1955 HOLT Harry G 79 Bournemouth 6b 182 (you were correct!!)
Hope it is of further help.Sue