The Barons Grantham were not the same family as the Robinsons of Rokeby. In fact, Thomas Robinson Baron Grantham was "short and fat". His contemporary and namesake Thomas Robinson of Rokeby was "tall and thin"!
Sir Thomas Robinson 1st Baronet left no legitimate issue. So I believe the titles passed to his brothers in succession, before devolving on a distant kinsman.
Sir Thomas's only child and heir to what was left of his estate was an illegitimate daughter, Ann, who married William Pirner of London in 1774. They were among my ancestors.
The Robinsons of Rokeby claimed descent from the Robertsons of Strowan in Perthshire, Scotland. I have some more information on the Robinsons of Rokeby, but nothing I can see connects them to the Isle of Wight.