The only likely relative I have found besides wife and children/stepchildren of Peter is Louis of Mendon. Peter's sons Louis and Charles were living with him on the 1880 census listed as nephews, but of course this doesn't necessarily mean Peter and Louis were brothers, they could have been cousins and it was simplest just to tell the census taker they were nephews. But I have always presumed them to be brothers for the purposes of researching. I have Louis' naturalization record, probate documents after his death to settle his estate, and many land records in addition to some birth and death records for children. The 1870 census for Louis is still a little strange to me because the children's names do not match but I have not been able to figure it out.
The Grafton, VT death record has never been one I put much faith in. I have no information of Peter ever being in that area of Vermont, and I believe the only reason it has been referenced is because an early Stebbins researcher grabbed it and stuck it to him and that family tree is still found everywhere online. The same Stebbins family tree is also the one that lists Louise Stebbins as dying in Jamaica, VT in 1902, but I have since found her death record in Farnham, QC in 1877 so the legitimacy of much of that information is in doubt.
I certainly welcome another interested researcher looking at this along with me though. I am certain there are documents somewhere that will help solve the mystery of Peter's origins, but it just hasn't been located yet. By the way, I am connected through my great grandfather Peter born in Rutland in 1858. Please have a look at my tree if you haven't come across it already and if you come across anything in your research please let me know.
http://trees.ancestry.com/tree/53443520/person/13510949246