He didn't mean one wife at the time (i.e., no polygamy). He meant one wife in the sense that there were no OTHER "wives" (i.e., women with whom they had "joined" in the flesh)... living (because death ends the bond)... or who had not otherwise been "released" due to fornication/adultery.
Thanks for this. I love hearing a different perspective. Something that seems obvious, but has been glossed over because of a commonly held interpretation.