Why don't you post their street address so that other can guide you?
What year did they reside there?
It is possible they traveled to another NYC county to attend church.

You might also search an historical directory.
Usually churches are listed by denomination in the front pages.

You could also email the Queensborough Library reference desk and provide them with details that I asked you for.