Do you expect someone to actually ask you directly if you attended the memorial ?
If so, I wouldn't give them power to make you stoop to lying.
I think if I were asked directly and I was planning on fading and not seeing them anymore anyway, I'd pause and then say:
Them: So...did you even attend the Memorial this year ??
Me: (smiling) " Don't get me wrong but I'm just sort of curious as to why you would ask me that?"
Them: Oh....I didn't see you at ours so I figured you didn't go or maybe went to one somewhere else?
Me: Oh...Ok I guess that makes sense, (chuckle) for a minute there it was almost as if you were checking up on me.
Them: So....did you?
Me: Pfft... have you ever known me to miss a Memorial?
Them: What Hall did you go to?
Me: (over your shoulder with a wink as you walk away ) Ha !...you're really too much sometimes.