Frankly, I don't see how contradicting someone when they tell you where they are from, and demanding a "real" answer, could be seen as anything other than extremely rude. Like, accusing people of lying at all is rude, let alone about something as conversationally trivial but personally important as one's place of birth.
If there weren't a racial element, the conversation would be baffling. "Where are you from?" "Chicago" "No where are you REALLY from" is a conversation only ever had between a white person and a non-white, because it would not make sense to randomly accuse someone of lying about their place of origin unless you're a racist trying to make someone feel out of place.
So, when I've had the same conversation hundreds of times in my life, but I'm white, what are the people trying to do to me?
Really now? Because you're white, people rudely insist that you're "really" from England and thus do not belong?
No. People ask
where I come from when they hear my non-Anglo name. Now, at this point, I know what they are asking me. They are not asking me: what city in Australia were you born in? They are asking me: what is your ethnic and cultural heritage. I tend to answer both questions in my response, and so either way, they are satisfied.
White people, even in white-majority countries, are often asked
where they came from. If you are white but with a non-Australian accent, I would ask you
where you came from. (I'm sure Canadians get a bit pissed off when they are mistaken for American, but that's the Canadian problem for not having a distinctive enough accent). If you are white with an Australian accent but with a marked name (anything other than bog standard Anglo-Celtic), I would also probably ask your ethnic and cultural heritage.
When people ask, I rarely think they are asking because they think I do not belong. They are asking because it is likely that my recent ethnic and cultural heritage is not Australian, and they're curious, or perhaps not even curious but are making polite small talk.