I suspect this elders' words would be significantly different in the presence of other elders.
However, he might just try to deflect the other elders from forming a witch hunt against you.
Most elders (in my experience) would rather not be bothered with complex situations, and an "apostasy" committee with someone who knows his stuff would be quite complex.
I'd put your odds of just being left alone at better than 50% (assuming you don't do something they can't ignore, like decorate your house for Christmas or get caught posting "apostate" flyers on cars in the KH parking lot).