They're not.
I've seen/heard some of the attitudes people have, especially towards non-white personnel in the service industry. It can get pretty bad. When I was working as a cashier as one of my first jobs fresh out of high school, I even had one old lady confide to me: "The Asians are following me. They've been following me since the train station." Right behind her were an Asian couple. It was arguably one of the most awkward moments in my life. I mean, just what the fuck do you say to that sort of shit? Especially when people of the same race she's talking about are right behind her within earshot... It's not like you can disabuse a complete nutjob of a customer of their silly notions when you're in the service industry either.
As for my background, I'd say middle class, although I did go to a private school, which the Premier's son attended and also the son of the CEO of Wesfarmers. :v Also, a current state govt. minister used to work there as a teacher (Peter Collier). So I've had a decently priveleged upbringing, Redezra.
That said, minorities, like indigenous Australians, can be equally racist back, even without cause, which I've personally experienced in the centre of Perth. So, yeah, it's not all sunshine and lollipops.