If you hide or cower away, they think they are doing the right thing, because you are hiding, feeling guilty, are ashamed, or whatever. They don't even have to consider their actions. What if you were severiouly depressed, and a little smile would have made your day? What if you were mentally ill, and their cruelness, caused you to go home and hurt yourself.
If you smile, wave, show them your new tan or boobs or car or hobby or whatever, it leaves them feeling confused. You aren't supposed to be happy when you leave the borg.