That's more or less the argument used by Samuel Johnson to 'refute' the subjective idealism promoted by Berkeley (well I'm afraid kicking a rock and being in pain afterwards is still fully compatible with idealism). Personally I don't think that what you say is enough to offer the argumentation needed to strongly 'anchor' our knowledge, likely once and forever.
The problem is that one can very well accept that truth is correspondence with observed facts and still make the common sense observation that we can nonetheless still be far from Truth (the-reality-in-itself), in other words truth can be very well domain-based and we not necessarily close to Truth. Let's take the movie Matrix as an example: before being freed from the Matrix Neo has no problems to admit that agent Smith is a human being, that he eats meat at lunch and so on. But after being freed he definitely accepts a totally different 'language game' when he visits again the Matrix even if he still accepts formally for example that it is raining in the city he once believed is his home and so on (in this new paradigm he solved also the puzzle of why he could not open his mouth when agent Smith asked him what use for a mouth if he cannot use it). Finally the conclusion is that iwhat happens in the Matrix is at least less real than what happens in the world of Morpheus at altri and of course that truth in the Matrix is 'far' from truth outside the simulated reality.
Ultimately all we accept as knowledge is 'laden' with the realist theory of perception which is the main paradigm at the moment but unfortunately we still lack the strong justification to think that this theory is the ultimate word or that it cannot be enriched in non trivial ways which to give us access to an 'outside the Matrix', possible to a Truth far from the truth we accept now. The problem here is not to claim that such an 'outside of the Matrix' is a reality (there is no such claim, as i already said i think, paradoxically for some, that we have actually more reasons at the moment to defend a sophisticated form of Realism as the first choice paradigm; the statement that, overall, science does actually approach Truth fully deserves the status of at least provisional knowledge) but to remain open to possible non trivial changes even at this level, basically nothing is outside possible rational replacement. Some scientists may easily dismiss such problems unfortunately Rationality do not allow us to do that.