The use and abuse of police data in protest analysis South Africa’s Incident System (IRIS)

South Africa’s Incident Registration Information System (IRIS) is a comprehensive, computerised database maintained by the South African Police Service (SAPS). In principle, it records all public order policing activity including all crowd incidents. While IRIS data is, potentially, a unique source for protest event analysis, considerable care is required. We aim to correct misunderstandings about the data advanced by academics and in the media, and expose its misuse by police chiefs and politicians. In particular, the incidents that IRIS reports are not protests, though protests can be found in the raw data. This article is based, in part, on records of 156,230 incidents covering the period 1997-2013.

How many protests are there in South Africa each year? A compilation of media reports provides one answer, but the South African Police Service's (SAPS) Incident Registration Information System (IRIS) can take us closer to a reliable estimate, because it contains considerably more records of protests. However, as we will show in this article, IRIS registers all public order incidents, not just protests, which means the data must be interpreted judiciously.
IRIS and its statistics are widely misunderstood and sometimes wilfully misused. One example of the former was a mistake made by Bilkis Omar, who, in 2007, confused SAPS reports of 'crowd management incidents' with protests. 1 A 2013 article by two journalists broadened the problem. Working from actual IRIS data, they claimed the police had recorded more than 3 000 service delivery protests in the preceding four years. 2 Looking more carefully at the data, we found they had incorrectly assumed that crowd incidents that had been assigned the IRIS 'motive' of 'dissatisfied with service delivery', were protests. 'Incidents' and 'protests' had been conflated yet again, and the IRIS category of 'motive' had been misconstrued. We return to this under Motives on IRIS, below.
While researchers' and journalists' errors were most likely unintended, the SAPS leadership have knowingly misled the public. They have done this, in particular, around the issue of violence. For instance, in 2014, Lt. General Elias Mawela, head of Operational Response Services (ORS), the SAPS division that included IRIS, told Parliament: 'Violent protest action escalated from 1 226 in 2011/12, and then in 2012/13 it is 1 882, and in the last financial year [2013/14] it escalated to 1 907.' 3 This statement elides 'unrest-related incident', one of two crowd incident classifications used in SAPS annual reports, with 'violent protest', but, as will be shown, they are not the same. Elsewhere, we have demonstrated that the same slippage was present in speeches by Police Minister Nkosinathi Nhleko and President Jacob Zuma. Disturbed by the way that statistics were being used to criminalise non-violent protests and campaign for increased funding, we exposed the matter for public consideration, and were damned by the SAPS for doing so. 4 This article arose out of our attempt to make sense of an enormous amount of IRIS data on crowd incidents. We were granted access to this after a request made under the Promotion of Access to Information Act (PAIA). 5 In total, the data represents 156 230 incidents, divided among 34 Excel spreadsheets. These covered the years 1997 to 2013, and there were separate sheets for 'crowd (peaceful)' and 'crowd (unrest)'. These two classifications, which IRIS calls 'eventualities', are also referred to as 'crowd management (peaceful)' and 'crowd management (unrest)', and they are aligned with the annual report terms 'peaceful incident' and 'unrest-related incident'. 6 We refer to events falling under these two eventualities as crowd incidents. 7 When using the data for protest analysis, one has to define and then find the protests. Regarding the former, we defined a protest as 'a popular mobilisation in support of a collective grievance'. 8 The latter task, of finding the protests, was extremely laborious, and we were only able to reach estimates through a careful reading of a large random sample of the open-ended notes recorded for each crowd incident. Elsewhere, we estimate that about 67 750 (43%) of the recorded crowd incidents were in fact protests. 9 In terms of scale, to the best of our knowledge, this means that IRIS contains more records of protests than any other publicly available and analysed police data in the world.
In this article, we limit ourselves to explaining IRIS; that is, its history, logic, concepts and biases. This is a necessary precursor to using its data for counting and analysing the protests it records. IRIS can be used for other purposes, and the article has relevance for anyone concerned with public order policing and crowd management in South Africa.
There are four substantive sections. The first summarises the kind of information that IRIS records, and in the process explains the differences between incidents and protests, and between unrest and violence. It also flags the importance of the relationship between IRIS and public order policing. The second section provides a brief account of their shared history. This paves the way for the third section, which examines IRIS's limitations. The most significant defect is its dependence on the capacity of public order policing to record crowd incidents. We then include, as an example of problems and possibilities of IRIS analysis, a section exploring motives, one of the many categories of data to be found in the spreadsheets. This highlights the predominance of labour-related incidents and the rapid rise in the number of communityrelated unrest incidents after 2004.

What iris records
Registration of incidents on IRIS is the responsibility of public order police (POP) units, and it is one of their major duties. 10 There are two principal logics underpinning what gets registered on IRIS or, more accurately, what should be registered. These are the recording of (1) all crowd management incidents, whether or not POP were involved, and (2) all incidents involving POP, whether or not these were related to crowd management. In a 2006 training manual, the former were described as 'primary tasks' and the latter as 'secondary functions'. 11 As we have seen, there are two classifications of crowd management: crowd (peaceful) and crowd (unrest). The distinction between the two is straightforward. If there was police 'intervention', the incident is categorised as crowd (unrest); if not, as crowd (peaceful 'Intervention' means the police exerted their physical influence in some way. It is not just about arrests, but would include, for instance, pushbacks, tear gas or rubber bullets. 13 POP do not have to wait until a crowd has actually been violent before intervening. Firstly, they are expected to act if 'life (and property) is in danger', and, secondly, 'if a national road is being blocked'. The blocking of other roads is left to the discretion of the operational commander. 14 This means that certain forms of non-violent disruption can trigger an intervention, and thus lead to a protest being classified as 'unrest'. The critical point is that an incident is defined as 'unrest' or 'peaceful' determined by whether the police intervened, not by whether there was violence. 15 From 1997, in addition to the primary tasks listed in IRIS, it had three main secondary functions, with a fourth added for the FIFA World Cup, which South Africa hosted in 2010. Public order policing and iris: an historical summary It is necessary to place IRIS and its development within the context of public order policing in South Africa. IRIS was established at a moment of insurgency and uncertainty in January 1992, just two months after the formation of the paramilitary Internal Stability Division (ISD). 24 The idea was to standardise information, linking this to a process of computerisation, 25 but it took five years to develop IRIS into a fully functioning system. On the one hand, public order policing was evolving. In 1995, following the Convention for a Democratic South Africa negotiations, numerous reports by the Goldstone Commission, the passing of the Regulation of Gathering Act (1993) and the 1994 election, the ISD was replaced by POP units. This entailed a process of transformation. 'Crowd control' was replaced by 'crowd management', which emphasised cooperation with protest organisers. There was retraining, a process of re-appointment aimed at weeding out racists (so we understand), and recruitment of black officers. 26 On the other hand, computers were not delivered until 1994, and there were teething troubles and debates about how to classify incidents. 27 IRIS generated some data in 1995, but less in 1996, and it was only from 1997 that it produced a full set of information. 28 Monique Marks, who in the late 1990s undertook ethnographic research with POP units in Durban, described the period from 1995 to 2001 as a 'golden era'. 29 In 2002, in the context of declining numbers of crowd incidents and a public outcry over crime, public order policing was relegated in importance. POP members were re-organised into Area Crime Combatting Units (ACCUs) and deployed to assist local stations. The number of officers was cut from about 11 000 in the POP units to 7 327 in the ACCUs, and it is likely that training deteriorated and the quantity and quality of equipment declined. 30 In 2006 there was further restructuring. With the aim of strengthening stations, SAPS areas, a middle level in its organisational hierarchy, were disbanded. The ACCUs were placed under national command, becoming Crime Combating Units (CCUs). Staffing was further reduced, to 2 595, and the number of units was cut from 43 in 2002, to 23. 31 Large parts of the country, including the whole of Mpumalanga, were left without any units. 32 For the SAPS, this cut was a serious blunder. As Omar commented at the time, there was a 'growing number and intensity of service delivery protests and riots', 33 and as Burger commented later: 'The short-sightedness of this decision was soon exposed when xenophobic violence erupted in March 2008.' There was a slight increase in CCU numbers,

iris data
This brief historical reflection is important for interpreting the IRIS data.

Crime prevention incidents
Other incidents Crowd incidents (peaceful and unrest) The most important reason for including the graph is to highlight the massive decline in all recorded incidents that occurred after the CCU restructuring of 2006. This affected crime prevention duties but, critically, from the perspective of protest analysis, it also affected crowd management. In the case of Mpumalanga, only four crowd management incidents were logged in the year 2007/8. 40 Day told us that for the three worst years, IRIS under-recorded crowd incidents by 20%-40%. 41 He added: 'They were policed but not recorded. We didn't have the capacity to get at them.' 42 In the above quote 'they' refers to crowd incidents and 'we' refers to the CCUs. This highlights a further problem with using IRIS data. Where crowd management is undertaken by forces other than POP, there is an increased likelihood that an incident will not be registered. 43 This has two pertinent consequences. For the first of these, it is necessary to factor in the impact of threat assessment, which has three levels. With Level 1 and Level 2 threats, POP units are, respectively, 'on standby' or 'in reserve'. It is only with a Level 3 threat that they are the 'primary role player'. 44 The threat level and response can change in the course of an event. According to the National Instruction, where there is violence, 'POP must take full operational command and stabilise the situation'. 45 In practice, because POP is less involved with Level 1 and Level 2 incidents, there is a higher chance that these will not be recorded, and, as a result, IRIS probably under-records peaceful protests. The second consequence is a geographical predisposition in the under-recording of incidents. In 2014, there were 28 POP units. These were stationed in the eight metros and 20 other major towns. A few extra POP units have been added in the last two years, but the situation has not changed significantly. If a protest occurs a significant distance from where the units are located, there is a higher chance that it will be covered by local police, or perhaps occur without a police presence. Hence, there is an urban bias in IRIS data.
According to Day, the proportion of incidents registered has been improving, with IRIS now missing only about 5% (most of these being in rural areas). 46 The SAPS's concern to secure additional funding for POP would be an incentive to improve the quantity of registrations, and its expansion should further increase reliability. However, one is wary of depending too heavily on this 5% figure, because we are finding media-reported protests on our database that do not appear on IRIS. 47 A further factor affecting assessments of the total number of protests is that the quality of data output is determined by the quality of data input. Unit commanders are instructed to ensure speedy capture of data. This is achieved by deploying at least one officer per shift to undertake the task. 48 Each POP unit has an IRIS controller responsible for checking data integrity and the system is, or was in 2006, maintained by the ORS Management Centre in Pretoria. 49 However, with evident frustration and concern, the authors of the Training Manual declare:

MANAGEMENT, CONTROLLERS AND USERS MUST HOWEVER REMEMBER THAT THE IRIS SYSTEM IS NOT A MAGICAL SYSTEM … The quality of the statistics … is wholly dependent on the quality of the information which is entered into the system. … [IRIS presents statistics] in a user friendly format [but] ... [t]his is not always a true reflection of what is occurring in an area, because the information [on] incidents [is] not captured correctly or not captured at all. 50
From reading the IRIS data sheets, we can also attest to the uneven quality of data capture.
From comments by SAPS officers and a government minister, we know there may be a variety of reasons for this, including the poor education of many officers, inadequate training and overwork. The SAPS is aware of the problems that exist with IRIS and we understand that there have been internal discussions about how it might be improved. 51

Motives on iris
This section examines a problem raised in the introduction, that of literal and uncritical interpretation of the IRIS lexicon. It looks specifically at motives. In the process we make an assessment of what crowd incidents were about, providing clues for further analysis of protests (a second phase in our research that is not considered in this article).
'Motives' are assigned to incidents using a dropdown menu. For crowd management incidents between 1997 and 2008 there were 60 options to choose from, and from 2009 onwards there were 72. An incident could be recorded with more than one motive, but until 2013 it was not obligatory to assign a motive to an incident. Indeed, 34% of all incidents recorded between 1997 and 2013 were listed as 'no motive registered', which is a significant obstacle for analysis. In practice, minimal use was made of the majority of motive options and Figure 2 just shows the 10 most frequently cited. It excludes, as do percentages below, no motive registered. The two most common motives were 'demand wage increases' and 'labour dispute', which together accounted for 25% of incidents. 'Sporting event' and 'social event' combined accounted for 10%. 'Dissatisfied with service delivery' represented 4% of the incidents. 'Solidarity', the third most common motive, is defined vaguely and applied inconsistently, and 'Forcing of demands &' (sic), the fifth most common (despite only being used until 1999), was also ambiguous. 52 Looking at the notes recorded for each incident, it is clearly wrong to assume that 0 5 000 10 000 15 000 20 000  . Two examples will illustrate the problem. The first, a crowd (peaceful) incident that occurred in the Eastern Cape in 2013, was an official event or imbizo that discussed service delivery. There is no indication that a protest took place. While it is likely that a higher proportion of crowd (unrest) incidents recorded as 'dissatisfied with service delivery' were indeed protests, there were exceptions. In a second example, from North West in 2009, there was a protest, but not over service delivery.
Possibly the recording officer conflated service delivery protest and community protest. On the other hand, there are numerous incidents assigned another motive that most of us would regard as a service delivery protest. Our conclusion is twofold: once again, one should not confuse an incident with a protest, and the assigned motives cannot be taken at face value.
That said, 'motives' can be used to gain some insight into the nature of incidents. We tried to get a sense of major trends by aggregating 'motives' into 10 broad categories. 53 As part of the process, we examined samples of incidents where the motive was absent or its meaning obscure. Clearly there is a high level of approximation in this process and a good deal of circumspection is required when interpreting the results. In the graphs that follow we only show the three most numerous kinds of aggregated motives. 54 For both of the graphs, the previously mentioned problem with data for 2007 to 2009 should be kept in mind.
In Figure 3, crowd (peaceful) incidents, there is a peak for labour-related and recreational, cultural and religious (RCR) incidents in 2010. This can be explained by, respectively, the 2010 public sector workers' strike, which in terms of working days lost was the largest in South African history, and the FIFA World Cup. There were fewer community-related events than labour-related and RCR events, and the trend for the former is flat. The picture that emerges in Figure 4, showing crowd (unrest) incidents is 4 000 3 000 2 000 1 000 0 1 9 9 7 1 9 9 8 1 9 9 9 2 0 0 0 2 0 0 1 2 0 0 2 2 0 0 3 2 0 0 4 2 0 0 5 2 0 0 6 2 0 0 7 2 0 0 8 2 0 0 9 2 0 1 0 2 0 1 1 2 0 1 2 2 0 1 3 quite different (though it must be kept in mind that there were far fewer 'unrest' than 'peaceful' incidents). Here there are fewer RCR incidents than either labour-related or community incidents and the line is flat. For labour-related incidents, there are two peaks, the one in 1998 and another in 2012, the year of the Marikana Massacre (although the Marikana strike itself was only a very small proportion of the total). The pattern for community-related incidents is more dramatic. Here there is a nadir in 2003 (as there is for labour-related incidents), followed by a strong upward trend, leading to a pinnacle in 2012. It is clear from the notes that, overwhelmingly, the community-related unrest incidents are protests, and that our database of media-reported community protests has a similar shape.
The graphs are revealing, for three main reasons. Firstly, the large number of RCR events underscores the fact that a high proportion of crowd incidents are not protests. Secondly, the media focus on community protests has drawn attention away from the high level of labourrelated action in South Africa. Thirdly, from 2004, there has been an explosion in unrestrelated community protests, reflecting what we have called a 'rebellion of the poor'.

Conclusion
For counting and analysing protests, data from IRIS has the potential to be a source of considerable value. The sheer number of recorded incidents made available to us is astounding, probably larger than anything similar elsewhere in the world. However, IRIS has been misrepresented by the SAPS and misunderstood by academics and journalists.
We have attempted to correct false impressions and have argued that its data needs to be treated critically and with care. IRIS exists to assist POP, and they are required to record crowd management incidents, not protests. We estimate that less than half of registered incidents are protests. Moreover, with the two main crowd incident categories, 'unrest' is 900 675 450 225 0 1 9 9 7 1 9 9 8 1 9 9 9 2 0 0 0 2 0 0 1 2 0 0 2 2 0 0 3 2 0 0 4 2 0 0 5 2 0 0 6 2 0 0 7 2 0 0 8 2 0 0 9 2 0 1 0 2 0 1 1 2 0 1 2 2 0 1 3 assembly of a number of persons without a joint goal or objective, after an incident or happening, e.g. a fire, accident or explosion.' It also explains (page 1) that 'crowd management tasks imply that the number of persons involved must be greater than 15 [but] this excludes a demonstration'. So, in contrast to 'gathering', the definition of 'demonstration' is aligned to that of the RGA.