2B or Not 2B? What Should Have Happened with the Canadian Long Form Census? What Should Happen Now?

Since Confederation, Canada has had a mandatory census with questions on religion, ethnic background, occupation, school attendance, literacy, infirmities, and much else (see Dillon 2010). Many of these questions are now on the long form census (2B) that since 1971 has gone to 20 percent of households. The remaining 80 percent of the households received the short form census (2A), which has basic questions on age, sex, marital status, family relationships, and mother tongue. In late June 2010, six weeks before the forms were to go to the printer, the federal government of Canada announced without public consultation that in 2011 the census would consist of only the short form, which would go to all households. The previously mandatory long form would be eliminated and replaced by the voluntary National Household Survey (NHS).

The government has argued that the data will continue to be of high quality with less state coercion.¹ Yet there is insufficient evidence to be confident about the data quality, and substantial risk that the NHS will be a costly failure.² Even given the government position that a mandatory survey is overly intrusive, there should have been advanced testing of the voluntary survey approach. I discuss some limited options that might mitigate the risk going forward with a purely voluntary survey.

What Is the Risk?

The risk of the voluntary approach is that the non-response bias may be high. The people who respond may be different from those who do not. The mandatory Canada Census has a minimal non-response bias because the response rate is so high. But even an overall voluntary response rate of 80 to 85 percent (much higher than Statistics Canada expectations for the NHS of 65 to 70 percent)³ could admit substantial non-response bias for some census tracts and some groups, such as immigrants or those with low income, low formal education, or neither English nor French as a first language. Non-response bias is particularly difficult because it is invisible; it depends upon the characteristics of non-respondents, which the statistician of course does not observe. Any loss of accuracy and comprehensiveness in the 2011 data would also entail the loss of comparability across time periods and with the data of other countries (see Dillon 2010).

How have the government and its supporters tried to defend the position that the new data would be just as high quality?

The first defence was that the sample of responses would be at least as large because the number of forms distributed would be increased from one-fifth of the population for the mandatory long form to one-third for the NHS to allow for increased non-response. ⁴ However, this approach will not prevent non-response bias. As Green and Milligan (2010) consider this important aspect, I will not discuss it further here⁵ except to note that the Industry minister appeared to suggest that the problem could be solved using non-response weights drawn from the mandatory short form.⁶ But this cannot counter the problem of response bias associated with, say, those of low formal education, because the short form does not have such data. In addition, a major advantage of mandatory census data is that there is little reliance on the assumption that those who do respond within a subgroup are representative of those who do not, an assumption that is required for non-response weighting to be effective. (For a more detailed discussion, again see Green and Milligan 2010).

The second defence was that the mandatory approach increases the probability of lying by respondents. This does seem likely, in part because those who might lie would simply not respond in a voluntary setting. The question is how large this problem is compared to the non-response problem. My view is that the government should have prepared evidence on this question to bolster its position and to help preserve comparability of census data over time. This is something that could easily have been done.⁷

In the case of income data...