Why has the word "yellow" not caught for describing east asian like white for euro race or black african has?

It has caught on somewhat. I have heard the term twinkie in the same context as oreo before as I have heard "yellow fever" in the same context as jungle fever lol. I think it will eventually catch on. Although I hope not. I don't like the idea that we call native americans red, africans black, Caucasians white, Latinos brown, and asians yellow. I mean hello. Am I the only person who doesn't see these colors in the majority of people they describe?