Racial inequality remains a persistent feature of American life. Despite the prominent place the idea of equality holds in the tradition of political philosophy, we remain without an effective conception appropriate for the experience of racial inequality. In this paper, I re-frame debates around equality and egalitarianism by reflecting on some of James Baldwin’s more strident arguments in his 1965 debate with William Buckley. I suggest he presses two complaints that are fundamental to racial inequality: the complaints of democratic distance and of disaffection. I then argue that while contemporary egalitarian theorists such as G. A. Cohen, Ronald Dworkin, Richard Arneson, and Elizabeth Anderson all claim to have isolated in their work a preferred conception of equality, they are unable to respond to Baldwin’s complaints, thus unable to effectively address the experience of racial inequality. I then leverage Bernard Williams’s distinction between a technological view of equality and a human view alongside his writing on imagination to offer a framework for meeting the moral demands that arise from taking the experience of racial inequality as fundamental to considerations over social and political equality.