The best place to start might be the Police Gazettes. These are held by Archives NZ:
http://archives.govt.nz/sites/default/files/Police_Gazettes....There is the following birth for 1886 which matches the the name above:
1886/13317 Winchester William Charles Walter /Mary Jane /William
I have a vague recollection of being told that Otago prison records (or maybe court records?) were lost in a flood of some sort? Not sure if that is correct or not. Most records of that nature, being governmental records, will be held by Archives NZ.