COMMENTARY

 

Rejoinder to Mkhatha et al. (2024): Problems and concerns with the 2022 South African census

 

 

Tom A. Moultrie; Rob Dorrington

Centre for Actuarial Research, Faculty of Commerce, University of Cape Town, Cape Town, South Africa

Correspondence

 

 

We welcome this opportunity to engage Statistics South Africa (Stats SA), even if only on a subset of the substantive issues we have raised^1,2 about the data from the 2022 South African census. Many of the issues were first ventilated in a memorandum sent to the South African Statistician-General in November 2023. Nevertheless, the nature of Stats SA's response still is overwhelmingly one of simply asserting that our concerns are misplaced or unfounded.

The litany of problems mentioned in the first six paragraphs of their response is noted. Each of these could have materially compromised the census results in both predictable and unpredictable ways. Faced with these challenges, it is perhaps not surprising that the undercount in the census was so great. We reiterate that, given the impact of the pandemic and its associated lockdowns on the functioning of all organisations, the decision to implement the census when they did was ill-advised. Even not knowing at the time when the pandemic would have run its course, it was unwise not to delay the census to October 2023, or even October 2024.

 

The balance equation

Stats SA's response³ either misunderstands or misrepresents our analysis based on the 'balance equation'. Fundamentally, and wrongly, they suggest that we use the balance equation to determine an alternative set of estimates of the population.

We do not. The analysis permits the identification of inconsistencies arising from the components of the equation (unpacking the "error of closure" to which Stats SA refer). We argue that the magnitude of differences cannot be explained by errors in the numbers of births and deaths, which we clearly acknowledge and point out is, relatively, small, so must be due to either errors in the 2011 census (and by implication also, substantially, the two previous post-apartheid censuses to the extent they are consistent with the data from 2011), and/or migration, and/or the 2022 census results. We note that as the difference/residual between the 2022 census and projections must imply a pattern of migration that is inconsistent with what would be expected (particularly at older ages), the implication is that the 2022 census estimate is too high. (In passing, try as we might and notwithstanding the fact that the published numbers have changed slightly between different published versions of the data, we cannot find any errors in our derivation of the numbers of immigrants from the place of previous residence, and are forced to assume that Stats SA's observation that the number of immigrants derived from these data is over 1 million is an error on their part). However, although we do not suggest that either 'our' or Stats SA's 2022 mid-year population estimate (MYPE) projections are acceptable alternative estimates of the population at the time of the census (both, to different degrees, appear to miss what could be migration of people aged 20-39, for example), we do argue that these projections of, for example, children, particularly aged 5-9, and those aged over 50 make more logical sense.

From this, and without producing an alternative estimate, we do suggest that the 2022 census could have overestimated the true number by "approximately one million". In this regard, it is instructive to note that interpolation from the 2024 release of Stats SA's own MYPEs produces an estimate that is nearly one million (960 000) below the census estimate at the census date of 2 February 2022. We note further that there has been a significant change in the MYPEs at provincial and particularly sub-provincial levels since the release of the census results. This has produced a number of curiosities in district-level population estimates. (As an important example, the implied population of the City of Johannesburg in February 2022 is some 16% higher in the new series of MYPEs than revealed in the 2022 census).

Such anomalies feed into the equitable share formulae used to allocate revenue and resources.

We do not presume that a cohort-component population projection from an earlier census will produce a superior estimate of the population at the time of the census than a successfully run census and Post-Enumeration Survey (PES).

Unfortunately, as we have expressed, we have doubts about how successful the 2022 PES was.

 

The Post-Enumeration Survey and small-population estimates

As we highlighted in the Technical Report¹, it is probable that the sample size of the PES was too small to accurately adjust the actually enumerated population in the census (of 36 million) to the final estimate of 62 million. This manifests in the severe anomalies in the estimates of the population of small groups and in small areas, some of which Stats SA concede is "convincing".

And yet, even in regard to the anomalous estimates we highlight, Stats SA erects strawmen to cast doubt on the issues raised. Of course, the lEC Voters Registration data are imperfect (although we did restrict our analysis to comparisons to the census population over 18) and may not fully capture the extent of migration. However, the population aged 18 and over in the Richtersveld local municipality more than doubled (apparently) between the censuses, while the IEC data from two local government elections held very close to the census dates show an increase of less than 5%. It is difficult for Stats SA to explain such results with generic expressions of (already acknowledged) concerns as to the weaknesses of such comparisons.

These census numbers defy common sense.

Likewise, they strawman the analysis of the Beaufort West town and local municipality, suggesting that satellite imagery of dwellings might hide increased numbers of people living in each dwelling. Hypothetically, that might be so. However, Stats SA's own data (in a report that no longer appears on their website) for the municipality indicate that the average number of people in a household in the municipality was unchanged between 2011 and 2022, at 3.8 persons.⁴ In other words, that simply cannot be the explanation because, if the average household size has not changed, and there are insufficient new dwelling units visible in satellite imagery, the purported growth in the municipal population is logically incoherent.

In preparing this response, too, we note that Stats SA is not above practising legerdemain. Their response makes much of the difference between the preliminary and final estimates of the undercount, and goes so far as to insinuate bad faith on our part. However, on the day the census results were released (10 October 2023), we each downloaded all documents and files made public on Stats SA's website, including the report on the PES. In order to respond to Stats SA's rebuttal, we have found that the version of the PES report that we downloaded has been replaced with a newer version, 4 pages longer, and including the tables they mention. At no point did Stats SA alert users to the new version or make it clear that the first version had been superseded.

Having now been alerted to the new tabulations, we find that they amplify our initial concerns. While the national estimate of the undercount shifts from 29.6% to 31.1% between the preliminary and final estimates, that of every population group narrows by between 5 and 37 percentage points. These changes would appear to be due largely to imputation of a very high proportion of people with undetermined/unknown population group enumerated in the census. In this regard, Stats SA has never revealed the extent of imputation of key variables or the processes and heuristics used in imputation, nor have they made data available to users that will allow investigation of the impact of the imputation procedures (by, for example, including edit flags in the 10% sample released to the public in August 2024, as was done for the 2001 census).

Finally, inter alia, probably as a consequence of this imputation, with the release of the 10% sample, we have identified a large number of worrying aspects in the weighting and the construction of that sample. As was the case previously, we will share these with Stats SA before publishing. Extreme deviation of weights in the 10% sample within the clusters used to derive the adjustment factors from the PES reinforces our contention in November 2023 that the data are not fit for purpose.

 

References

1. Moultrie TA, Dorrington R. The 2022 South African census. Cape Town: South African Medical Research Council; 2024. Available from: https://www.samrc.ac.za/research-reports/2022-south-african-census        [ Links ]

2. Moultrie TA, Dorrington RE. Problems and concerns with the 2022 South African census. S Afr J Sci. 2024;120(7/8), Art. #18585. https://doi.org/10.17159/sajs.2024/18585        [ Links ]

3. Mkhatha N, Djolov G, Iturralde D. Response to Moultrie and Dorrington (2024): 'Problems and concerns with the 2022 South African census'. S Afr J Sci. 2024;120(11/12), Art. #20465. https://doi.org/10.17159/sajs.2024/20465        [ Links ]

4. Statistics South Africa (Stats SA). Census 2022 municipal fact sheet. Report 03-01-82. Pretoria: Stats SA; 2023. Available from: https://www.datafirst.uct.ac.za/dataportal/index.php/catalog/982/download/12776        [ Links ]

 

 

Correspondence:
Tom Moultrie
Email:tom.moultrie@uct.ac.za

Published:04 December 2024