If you meet with them, you might slip and say something negative about the watchtower and it's doctrines and you're going to screw yourself.
These men are experienced in recognizing someone that might be an "apostate" and things will not go well with you..
Just fade and don't give them a reason to DF you.
Keep in mind you also have a wife and she will listen to these men more than she listens to you, the less you say the better.
Talking from experience .