Absolutely, it is more natural for some people to do so than for others. Some people are more apprehensive when confronted with someone who looks different. That doesn't make it the problem of the person who looks different. Or shouldn't.
It shouldn't indeed. Instinct ... reality: to humans it does. So in my encounter with one black man (we became friendly and conversed for a few hours) I had to recognize that he had a stereotype for an old white guy. After a couple of hours, we, in fact, discussed stereotyping and we both agreed we each had, as an individual, violated the other's initial stereotype and were more like than unlike.
It seems to me to be basic, evolved, ‘hardwired’, human ingroup/outgroup psychology. It has been observed in infants. Humans much prefer ‘perceived ingroup’ and will be biased in favour of it, however defined (‘likes same food as me’ in one experiment on infants).
Now, when you add into that basic framework that one (perceived) subgroup (one subset of all humans) rises above another, in status or economic or power terms, which tends to happen, over time, for a wide variety of complicated reasons, then especially if the perceived subgroups interact or coexist or merely rub up against one another (which also tends to happen, especially in a crowded world) I think you start to get patterns of privilege, inequality and unfair advantage/disadvantage.
As a white, straight, ‘western’ male, I automatically inherited a bunch of advantages I did not have to earn, because of patterns that had been established long before I was born.
All other things being equal and speaking generally, even a comparatively under-advantaged white person will still have a skin colour advantage over his darker-skinned neighbour and fellow human, and this advantage may be called a privilege because it is unearned.
Word up: the world is a place of competition. Advantages matter, a lot. Humans are essentially self-interested (albeit with strong social/reciprocal factors being involved).
In many ways, people utilise their advantages and privileges simply because they can and because it’s natural animal behaviour.
Layer on top of this what is called human morality.
Want to motivate someone to be good to you, share with you or care about your interests? Give them a good reason to.
Sorry if there’s no sugar coating on that.
Although in a way there is. It’s called mutual self interest. It doesn’t necessarily trump competition, when the chips are down, but it can often figure.
See also: game theory.
The alternative approach is to beat the perceived outgroup in battle (armed or political) or be able to viably threaten to (this would come under the ‘giving them a good reason’ heading mentioned above).