Fisherman,,once that has been pounded into your head its hard to unlearn.
However, the author of Matt (and Luke using Matt) utilizes the very popular parable of Israel being God's vineyard/field/household. (eg Isaiah chapts. 5 & 27, Ps 80) Though a series of parables, the author draws lessons utilizing this imagery. In Matt 21 Jesus says 2 parables, (21:28-46) the 2 sons in the vineyard and the tenants in the field in which the Jewish religious leaders appointed by God centuries previous are the target. THEY were the author's disloyal, faithless, workers/servants/sons. There is nothing to suggest the parallel parables in Matt 24 and 25 refer to anyone else. THEY, that is the Jewish religious leaders, failed to recognize the son's/master's arrival and were found sleeping, lazy, corrupt etc.
Its key to remember the authors' of the Gospels did not have a 2000 year disappointment in mind. They intended the "delay" to refer to the long period of anticipation of Yahweh's return by his Messiah. For them all the events referred to in the parables, judging of religious leaders, the signs, the revealing of the Messiah's power etc. were to happen in their generation not in the distant future.
It was centuries (and possibly an edit or two) later that the parables seem take on new meaning wherein the delay wasn't the wait for the arrival of the Messiah it was the 2000 years of disappointment that followed. The harvest wasn't imminent, it was thousands of years later. The parables lent easily to that misapplication that as long as you disregard the specific application of the parables to the Jewish leaders and the imminent judgement expressed in the same context.