LIke you, having lost friends and family to a cult, I have a preoccupation with not enjoying losing anyone from my life.
Some, however, are not healthy for us to be around. Hard experience tells us that. We might forgive it for a while, but eventually there can come a point where we [have to] let it go. If it is a greater drain on our energies than we can affford then we will just get sucked down with them, and some people truly are energy-vampires!
There are all sorts of methods that this comes packaged as, including comments intended to hurt. After all, what kind of friend intentionally hurts another (and the intent is key, here)? It's harder to forgive than most other behaviour. Nonetheless forgiveness doesn't mean that continued company is a desirable outcome.
You perhaps need to decide if that point has been reached, or not.