You have gotten a lot of good advice from people here and that is to refuse any visits from the elders! You are on their radar and they are not there to "encourage" you.
As you said, there are people there more in need of an "encouraging" call. And, by the way, they are not interested in the single women there except to target them to do favors for all the users. These women should get the hell out of the Witness religion and their depression will improve. I can attest to this.
Meanwhile, heed everybody's advice and refuse intrusive visits. If you are a "nice" guy who hates confrontation and to be seen as disagreeable, STOP answering the door when they come by. START screening phone calls and do not answer texts. It may be necessary to finally block their numbers if they persist.
If you do not do these things you will be prolonging the pain.