Hi Michael, hope this helps... most likely you've seen some of the links...
Here is a site provided by the VT GenWeb... excellant with GPS, Towns, & a big green map in the upper right hand corner so you can see the realtivity of the towns -west-east-.http://www.nekg-vt.com/Cemeteries%20of%20VT/orleans.htm
If he mustered out as the entry here from Ancestry.com CW Soldier records state, "Mustered Out Company H, 15th Infantry Regiment Vermont on 5 Aug 1863 at Brattleboro, VT." he had a long trip home to Orleans county all the way to the Canadian boarder by the time he died on Sept 10, 1863.
Are other CHURCH family buried in any one cemetery? I did not find him on FindAGrave, but there are many other Church in Orleans county.
The Vermont Old Cemetery Assoc. may be able to help if no one here has an answer.http://www.voca58.org/
Joshua Church is also found on this site, if you get a pic of his stone, you might consider supplying them with a copy! (search by soldier name)http://vermontcivilwar.org/cem/cemetery.php