Let me also ask you this.
Under your theory, no one can tell someone who to speak with and who not too.
Let's say that you are a parent and Timmy down the street you feel is a bad influence and you tell your son not to speak with Timmy because you think he will corrupt your son. Would that be a violation of your son's rights?
Or let's say that your wife slept with someone else and you are so hurt that you tell your grown kids if you speak to your mom again, don't bother speaking to me, because you chose her over me. Would the father be infringing on this children's rights?
There is no compelling governmental interest to ensure that people speak with each other. Even in child neglect laws, the only requirement of communication between a parent and child is to ensure the child's basic necessity. If the parent never says I love you or never says how was school, there is no compelling interest for the government to enforce that.