Your subject line uses the term "stillbirth", but your post suggests that the baby in question was born alive. Technically speaking, a stillbirth is where the baby draws no breath and different procedures apply.
Until 1927, there was no registration of stillbirths. After that date, stillbirths were registered as such (i.e. not in the same way as live births). They do not appear in the GRO birth index - the register is a separate one - and applying for copies of the certificates may be problematic. There would have been no death certificate (on the basis that if the child was not, officially, alive it could not have died). Churches do not record births and deaths, only baptisms and burials. Only living infants can be baptised so there could have been no ceremony performed for a stillborn one. The standard burial practice for stillbirths at that time would have been to slip the body into the coffin of the next available burial or bury several babies at the same time in a public grave. The burial should be recorded in the burial register of the cemetery or church yard concerned but there is unlikely to be any grave marker.
Strictly speaking, the circumstances you refer to would not classify the infant as stillborn. If the twin lived, even if only for a few minutes, it would be deemed a live birth and the normal birth and death registration procedures would have applied. However, it was common for midwives and others attending such births to say that the child had been stillborn, to save the parents (and themselves) the extra administrative burden. Well-meaning, but misguided.
Hope this helps
Caroline