I have turned up an interesting clue to the Haddam Rays. In the History of Haddam, the following is found: "About 1710, James, Peter1, and Joseph Ray, three brothers, Portuguese, settled here, immediately from Narragansett. James (who died over 100 years of age) was f[ather]of James, Benjamin1 and Joseph; Joseph was f[ather] of Isaac, Nathaniel, Jeremiah, 4, Joseph,1, Timothy,1, Elisha, 3, and Jacob, 1."
James did not live to over 100 years of age. However a Simon Ray of Block Island, RI, did live to over 100 years, dying in 1737. Narragansett is one of the traditional points of departure for ferry boats to Block Island. Simon Ray was from Suffolk.
I also found this: "Simon Ray from Suffolk came with his parents to Braintree as a boy in 1640. His father died the following year. In 1661 Simon departed with his mother and step-father to Block Island off Rhode Island where he was one of the original settlers. He lived to be 102, dying there in 1737." http://www.selectsurnames4.com/ray.html http://www.selectsurnames4.com/ray.html
I am going to follow this further, but I think the living to over 100 years coincidence combined with the Block Island/Narragansett coincidence solves the mystery of the origins of the Haddam Rays.
If so they are not Portuguese (not likely in any case), but from Suffolk, England.
Simon Ray was justice of the peace for over 40 years.
Update: This seems to be a dead end, Simon Ray is not the father of James and Joseph according to his bio in Genealogies of RI Families.