I think also that some of what the Watchtower taught us (and maybe a few liberties taken by Hollywood movies, like The Ten Commandments and The Prince of Egypt) warp our memories of the Exodus narrative. First of all, the story of "plagues" and the exodus itself is believed to be a liturgical drama and not history by both Judaism and Christianity.--See the footnotes, introductions, and additional materials in both Etz Hayim: Torah and Commentary, the official Bible of the Conservative Jewish movement and the NABRE, the official Bible of the Roman Catholic Church in the USA.
Second, in the Tenth Plague, "children" of the Egyptians are not killed by God, only firstborn "sons" die--and these are generally the eldest in each family (a significant point). The Book of Exodus opens with a pharoah of Egypt demanding the murder of infant "sons" of Israel, but allowing the girls to live. (Exodus 1:22) So in the final plague, God attacks Pharoah (who claims to be a god) by killing his "firstborn," or his next in line to rule, another "god," so to speak.
In Judaism, "firstborn son" means "heir" and refers to the eldest male who inherits the same rights as the father. For example, Jacob offer Esau a bowl of stew in order to get his right as firstborn, and later was helped by his mother to seal this with stealing his brother's blessing from his father. (Ge 25:29-34; ch 27) And it often means a royal heir, not literal child at all.--See Psalm 2:6,7.
This is what what was meant at the very beginning when God told Moses to warn Pharoah using these words:
You shall say to Pharaoh, ‘Thus says the Lord, Israel is my firstborn son, and I say to you, “Let my son go that he may serve me.” If you refuse to let him go, behold, I will kill your firstborn son.”--Exodus 4:22, 23, ESV.--Compare Ex. 19:6.
Thus, the drama unfolds with the Hebrew God striking Pharoah a blow to his heir, as promised via Moses at the first revelation at the Burning Bush. It was actually the first warning Pharoah ever got when Moses first visited him before even the first plague broke out--and thus foreshadowing how the story of the plagues end, with Pharoah losing his heir to the throne, not with literal children dying all over Egypt as you often see playing out in Hollywood movies or perhaps the way Jehovah's Witnesses mistakenly understand the text.
Finally, by definition "theodicy" is actually not a debate over why God would punish another with death if they deserved it. In the above drama, the pharaohs of Egypt are guilty of infanticide and thus God repays them by killing their firstborn, so to speak. (Again it's a liturgical drama, not a historical piece.) Why does theodicy not work here?
Because theodicy is not questioning why God brings justice upon the wicked, but why God allows evil if God can obviously do something in the first place. That is the opposite of what happens in the Ten Plague story: Pharoahs kill Hebrew sons, God kills Pharoah's sons. That is a story about God actively bringing justice for all to see.
Theodicy is questioning why doesn't God act at all, if there is a God. Why is there suffering? Why does evil exist? If God is good and omnipotent, then how can all this other stuff be? Why doesn't God act? Where is God? That is theodicy. For that, we have the Book of Job (which by the way, does not answer the question--it merely adds more questions and recognizes that humans cannot answer it).