hoser - "I can go on vacation to the same place, swim, snorkel, enjoy some shows, drink until I puke my face off, and it would be overlooked, but if I swing a hammer so a poor person can have a house, I could be in trouble with the congregation..."
Pose that very sentence in question form (with the intro "let me get this straight", and the outro "seriously?") to the next person who advises you not to do it on those grounds...
...and watch the expression on his or her face.
If they say "yes", they'll look like assholes and idiots, and they'll know it (unless they really are idiots)... but... they won't want to say "no" either, because of the typical loyalist's knee-jerk rejection of anything not WT-centric.
The internal conflict will be readily apparent on their face (and hugely satisfying to watch), and if it all goes down in front of a group of people, so much the better.