Many accounts of the contemporary structure of racism argue that we live in an age characterized by color-blind racism. In many of these accounts, color-blind racism is discussed as a distinctly contemporary phenomenon, brought on by the rising regime of neoliberalism in the late twentieth century. This article problematizes this periodization, arguing that the first, developed, color-blind racist philosophy was, rather, developed by sixteenth-century Spanish jurists seeking to develop an international legal framework to justify—in universal, humanist, and color-blind terms—the colonial domination and exploitation of the Caribbean and the Americas. Through a careful reading of the work of Francisco de Vitoria, I explain how the creation of a color-blind system of universal human rights—specifically, the “universal rights” to travel and commerce—operated to uphold systematic white and Euro supremacy through color-blind discourses. I thus argue that in order to understand contemporary manifestations of color-blind racism, it is necessary to understand it as a consequence and development of this earlier colonial history.