As children learn their mother tongues, they make systematic errors. For example, English-speaking children regularly say mouses rather than mice. Because children’s errors are not explicitly corrected, it has been argued that children could never learn to make the transition to adult language based on the evidence available to them, and thus that learning even simple aspects of grammar is logically impossible without recourse to innate, language-specific constraints. Here, we examine the role children’s expectations play in language learning and present a model of plural noun learning that generates a surprising prediction: at a given point in learning, exposure to regular plurals (e.g. rats) can decrease children’s tendency to overregularize irregular plurals (e.g. mouses). Intriguingly, the model predicts that the same exposure should have the opposite effect earlier in learning. Consistent with this, we show that testing memory for items with regular plural labels contributes to a decrease in irregular plural overregularization in six-year-olds, but to an increase in four-year-olds. Our model and results suggest that children’s overregularization errors both arise and resolve themselves as a consequence of the distribution of error in the linguistic environment, and that far from presenting a logical puzzle for learning, they are inevitable consequences of it.