I am a direct descendant of Colonel Miles Cary, founder of the Virginia Branch of Cary in America and WiLlam Cary 1492 Bristol England.

The information I have on Castle Carey -
1170 -Adam de Kari was Lord of Castle Karry in Somerset - 1193 listed Karie of Torre Abbey as tenant in chief in the Doomsday book . The name appears spelled as "Kari" and "Karry" then later as "Cary" and "Carey"

The name was derived rom the manor of Cary or Kari in the Parish St Giles in Heath near Launceston.