So here is something else that helped me wake up -
The Elders Manual - Shepherd the Flock of God contains a special caveat for Elders who have committed adultery some time ago (Not sure how many years, 'some time ago' is, but probably based on a complex formula of donations and # of assembly parts given.)
But basically, they can just get a talking to by 2 brothers and then they can keep on keeping on.
That's the is short of it.
So this is definitely a good old boys club rule to protect the club. If you are doing other things then it's the judicial ax for you.
Funny in the below capture - where is the scriptural support for this ?
TL/DR - If you are going to cheat on your wife, make sure you are appointed and don't tell anyone once you do cheat for at least a few years. This tricks the Holy Spirit into not revealing the matter.