The goal of this paper is to problematize the use of sex categories in medical contexts. We question the benefits of categorizing all individuals as either male or female in medical contexts and argue that we should focus instead on the relevant sex-related properties of patients. Contrary to what many people believe, the classificatory system by which sexed bodies are neatly divided into male and female is anything but clear. An abundance of evidence shows that a binary sex system does not accurately describe the reality of human bodies. Given the complexity of sex with its many markers, variations, and combinations, why is medicine still based on the assumption that there are only two sexes? Why is binary sex still systematically used as a proxy in medical contexts, even when it doesn’t help, but can actually hinder diagnoses, care, and treatments? The complexity and heterogeneity of sexed bodies is critical in medical contexts. We argue that the use of female/male categories overlooks and obscures this complexity and variety, thereby resulting in a variety of harms, poor health care, oversimplification, and over-pathologization.