It wasn't happening that whole families left when I stopped attending. But I remember meeting at Flipper's Tahoe-fest, a whole family that left together when the husband stayed in just long enough to make sure his family woke up. It was a wonderful thing.
I think it would often be true that one spouse is more ready than the other, and the kids are just ready anytime to drop that crazy cult.