Pretty much all the developments to come out of the industrial revolution in Britain in the 1700s and then from the colonial powers in the 1800s, into the mid 1900s were made on the back of oppression.
Of course the worst of these was the years of slavery endured by generations of black Africans. Without drawing a direct comparison, many working class people suffered as wealthy land owners and industrialists developed big businesses manufacturing goods and building infrastructure. People worked long hours for little pay in very dangerous conditions in mines, mills, building railways and so on. Many children were forced to work at very tender ages. Education was minimal. Health care all but non-existent. Many business owners ran their own shops and currencies, trapping people in a cycle of poverty and, in a broad sense, enslavement.
There is still a legacy in the UK in terms of class, privilege, entitlement. It's not as distinct as it once was but it is woven into the fabric of the social structure. To suggest that the impact of slavery or apartheid will not permeate the culture of those whose past is directly wedded to this history is ridiculous.
The questions are how and when do things move on? When does a society accept that whatever the reasons as to why one is where one is, what really matters is how the future is made better? When should the past be left in the past? Was everything that happened in times of oppression always bad? When does pragmatism and optimism trump indignation or justice over past grievances?
I don't know the answer but I am sure that chucking a turd at a statue proves that some aspects of society are not prepared to move on. Should they?