No, it just depends on what you consider "meaningful". Non-racists see benefits in multicultural societies; Racists see pollution in a society with anything less than 100% of their race. So is a shift from 'about one in five' to 'about two in five' strangers on the tube don't look enough like me for my tastes 'significant'?
If you believe that "racists" believe in an imagined past where London was 100% white in 1961, then it speaks more to your caricature of who you think is racist.
But to answer your question, yes. A shift from 20% to 40% non-white is literally
doubling the percentage of non-white people. It's difficult to believe you think that's trivial. It would be difficult to believe you would find any similar demographic shift trivial. A mining town where female residents went from 20% to 40% would notice it.
And there were plenty of Afro-Caribbean, African, Indian, and South East Asian people in London in 1960 who were not 'foreign born'.
Unfortunately, I can't find ethnicity breakdowns (foreign born or not) for London for 1961, but I suspect it would be whiter than in 1991, which in turn was significantly whiter than 2011.