“If we were to wake up some morning and find that everyone was the same race, creed and color, we would find some other cause for prejudice by noon”