The old saying is the best defense against being charges of defamation is the truth, but notice it's a defense: the truth still needs to be proven in court if someone wishes to sue you in a case they'll likely only lose. You'd still need to prepare a defense, go to court, waste time/energy/$. Most people wouldn't be so petty as to sue over a matter they'd only lose, but some people with deep pockets would, just to try and get someone to feel some of the pain.
Depending on who the other person is, they may be considered a 'public figure' and hence have less expectations of privacy, and hence, less grounds to win a defamation case (in law, the idea is a person willingly entered into the public arena, and voluntarily sacrifices some of their privacy rights to do so). Eg Mitt Romney, or ANY politician is a public figure, and the burden of proof rises for them, where the accused has to show malice and actual disregard for the facts to cause them harm.
Another consideration is damages: the defamatory words have to actually harm the person financially (eg their business) to make it worth suing someone. If it's just insulting their ego, no judge would award an amount to make it worthwhile (although, see above).
Upshot is, freedom of speech is very broad in the US, and hence it's difficult to prove a defamation case, ESPECIALLY if truth is proven as a defense! If you have a strong case and feel it can be proven, I'd tell them to go pound sand (or I might Apologize, if I wanted to reduce the odds of being dragged into court to prove the obvious).
