Not Racism but just not used to us
Here in Japan, it has gotten better over the last few years however foreigners have had a hard time. The problems are things like, getting property, even renting, getting credit / loans etc. And a more apparent one - people sitting next to you on the train or bus.
I have been here for over 13 years now and have seen many things, but since I am a permanent resident I was able to get a mortgage, buy a house etc. But many other foreigners struggle.
Let's start with the trains. The empty seat next to a foreigner will probably be filled up lastly or in many cases never. What's the problem here? we don't smell !!. It's not really racism, my wife is Japanese and her family all accept me just like a Japanese person. It's mainly the fact that Japanese are not used to use. Even now 97% of the population is Japanese, it's not really a mix culture however more and more foreigners are living in Japan.
So basically, what the Japanese are not used to, or don't know about, they don't trust. However that is slowly changing for the better.