The JWs attend the memorial as "observers" but the only thing they are "observing" is other JWs "observing". They can't help it. There aren't enough annointed ones™ to go around to all the congregations, so there is nothing to observe other than one anothers non-participation.
If someone did dare to partake™ they would be the subject of gossip, most of it malicious, for the rest of the year until the next memorial™ came around. I remember one guy who started partaking™ and people openly started giving him a hard time at the meetings from then on, asking him, "Who do you think you are presuming you're annointed?"
Two others I knew who started partaking™ were (supposedly) later diagnosed with mental illnesses, and some JWs held the opinion that the illness was a "punishment from Jehovah for partaking unworthily".
W