Removing yourself from it is vital. You don't have to do it all at once if you are
worried about the wife rejecting you, but it can be done.
You might try just walking away from it cold turkey or you might start missing meetings
here and there. The best way to see what will happen is to start skipping now. See
what the wife and others have to say, then skip another meeting, perhaps Sunday.
Come back and let us know what you are doing, what they are saying. We can help
you to fade or just leave. It can be done.