I think the difference has to do with ostracizing former members. When a group does this, then whether or not they used "brainwashing" techniques to get peoplein, the people in have a tremendous amount of pressure being put on them to stay in no matter how badly they're treated.
The problem I face when it comes to studying this issue is that there is I'm having a hard time finding data on which groups do and do not ostracize former members.
Here's a list of some groups I consider destructive cults with information on whether or not they ostracize former members in paranthesis:
Destructive Cults
Scientology - Practices Enforced Ostracizism of former members (Disconnection)
Jehovah's Witnesses - Practices Enforced Ostracizism of former members (Disfellowshipping)
Synanom (drug rehabiliation center that evolved into a highly violent cult) - Practiced Enforced Ostracizism of former members (who were called Splittees)
Jim Jone's People's Temple (cult that ended in mass suicide) - Practiced Enforced Ostracizism of former members (even holding mock funerals for former members)
Can anyone think of other examples?