When did your grandfather come to America?
There should be some paper trail that he left behind. His arrival manifest would have a place of birth or last residence. If he was naturalized US citizen than his Intent & Petition will have his place of birth. Sometimes a Draft Registration Form for WW1 and WW2 can have a place of birth. You also might want to look into requesting his SS-5Form if he had SS#.