Greetings!
Since this is your husband and these are your feelings as they pertain to his conduct, you have the right and should tell him about anything that makes you feel uncomfortable. As your husband, he should be considerate of your feelings and even though something may be habitual he ought to make an effort to be more considerate of you.
Notice the above applies to anything.
With regards to Witnesses using the term "worldly" I do have to take the unpopular side here and defend the use of the term "worldly" by Jehovah's Witnesses.
Of course, there MAY be some that overhear witnesses using this term in their conversations or occassionally from the platform and may understand the negative connotations involved; however most people, that is everyone else sitting around the restaurant would not so understand those connotations. In fact, as has already been pointed out, they are likely to perceive it has having positive connotations or at least be confused since among normal non-JW people the term does have positive connotations.
All coherent groups (from the military to lawyers to street gangs) utilize forms and terms of language that are exclusive and shared by the group and helps to separate them from non-group members. One who understands a shares the same symbol-referent system is identifiable as being "in" with the group. Such language behaviour is common and unobjectionable and in fact useful.
As you probably know, JWs use the term "worldly" to describe everything from a non-JW to a JW that is engaging in certain behaviours or possesses certain traits. The opposite of the state of being "worldly" (or of THIS world/physical) is of course to be "spirtual" which represents the ideal. Thus (in theory) JWs strive to be spiritual people and not worldly people.
It is a useful term that used among JWs isn't offensive and is effective in communicating. Rather then being offended you should consider yourself as being welcomed and "in" enough with the group of JWs that they would use such term in your presence, without any kind of deference or making excuses to you. Obviously, they feel comfortable with you.
If you are offended then you should simply assert yourself and say something along the lines of:
I understand you guys are using your language and that you don't really mean any offense to anyone, but as a "worldly" person who knows that the term conveys certain negative connotations, it makes me feel demeaned to hear you use that term right in front of me. Would you please be more considerate in the future? Thank you.
they will get the message and should be happy to be more considerate in the future.
-Eduardo