And ha now who's going to be my new elder... gees with all these new elders the whole world might as well no my business!
Bingo, you hit the nail on the head! The old elders will tell your new elders why you were DF'd, and those elders will tell their wives, and the wives will accidentally let it slip to their friends, who will tell their friends/husbands, who will tell their friends/wives/children, until the entire congregation knows what you did, as well as some neighboring congregations.
One how is it right I got two family members as my elders?
You can get family members as elders, you can get an elder your age, heck you can even get an elder that is younger than you these days! And yes if you are Df'd and have a high social status in the congregation (ie family member of an elder) then you can expect to have an easier time getting reinstated. Sucks doesn't it?