No-one can force you. The sooner you say "no more" the better. It's hard to tell your family that you're not going to do it anymore, but they'll get over it. Your greatest friend is the word "NO". It's as easy as that. They can't drag you out.
I hope your not talking about a kid or a teen because yes they can "drag you out" I know, wen I said no I always got the" this is my house my rules if your not happy get out" So I did and my dad call the cops on me then he called the elders. My wife wen she was 16 she was kicked out of her house for not "obeying" her step father.