Inequality causes a variety of social ills, which give egalitarians reasons for concerns of justice. In particular, inequality is deemed to undermine people's fundamental moral capacity of self-respect. In this paper, we explore the complex relationship between inequality and self-respect from a philosophical and an empirical angle, arguing that a theory of justice should take both into account. To this purpose, we first clarify the normative objection to inequality from the alleged erosion of self-respect. Then, we elaborate on empirical findings showing the crucial role that 'relative deprivation' plays in the causal mechanism that connects inequalities to the erosion of self-respect. We conclude that this role is best understood in philosophical terms as a form of deprivation that affects the significance that people attach to the value of the choices they make.