RESEARCH PAPER

 

Reviewing the quality of technical reports in support of water use licence applications

 

 

Jurie Moolman^I; Reece Cronje Alberts^I; Francois Pieter Retief^I; Claudine Roos^I; Dirk Cilliers^I; Alan Bond^{I, II}

^IUnit for Environmental Sciences and Management, North-West University, Hoffman Street, Potchefstroom, South Africa
^IISchool of Environmental Sciences, University of East Anglia, United Kingdom

Correspondence

 

 

---

ABSTRACT

The South African water use licence application process requires the submission of technical reports to the responsible authority, which, anecdotally, have been argued to be of poor quality. The aim of this research was, therefore, to evaluate the quality of a purposively selected sample of 8 of these technical reports. An analytical framework was developed comprising 13 key performance areas (KPAs) and 20 related key performance indicators (KPIs) for 2 review areas, namely, completeness and substantive quality. This analytical framework has potential application to the evaluation of technical reports for water licensing applications in any jurisdiction, subject to appropriate modification. Considering the completeness of the technical reports, good performance was observed for the inclusion of all relevant documentation, technical information and public participation. Poor performance was observed in the determined water uses, description of the environment and key impacts. When considering the substantive quality of technical reports, good performance was observed for describing the proposed activity, socio-economic considerations and dealing with scoping. Weaknesses associated with substantive quality included dealing with significance, mitigation and public participation. To address the weaknesses in completeness and substantive quality of technical reports, we recommend that the responsible authority develop guideline documents for dealing with significance and mitigation, and that existing guidelines be implemented to ensure meaningful discussion during the public participation process.

Keywords: technical reports, completeness, quality, water use licence applications, South Africa

---

 

 

INTRODUCTION

South Africa experienced a slew of legal reforms with the establishment of a democratic government in 1994, with none more far-reaching than the legal reforms observed within the water sector, which at the time had to deal with past racial and gender inequalities when it came to access to water (Karodia and Weston, 2001). This water law reform was driven by the White Paper on a National Water Policy for South Africa (RSA, 1997), which included guiding principles on new approaches to be implemented to ensure sustainable utilisation of one of the country's most valuable resources. Two of the most noteworthy principles which mandated administrative adjustment were that "there shall be no ownership of water but only a right or an authorisation for its use" (RSA, 1997 p. 60) and that "any authorisation to use water shall be given in a timely fashion and in a manner which is clear; secure and predictable in respect of the assurance of availability, extent and duration of use" (RSA 1997 p. 62). These administrative adjustments, led to the promulgation of the National Water Act (Act 36 of 1998) (NWA: RSA, 1998) which was internationally regarded as the most progressive and advanced water legislation at the time (Mackay et al., 2003). The NWA abolished the discriminatory 'riparian principle' that allowed the use of water which resides in the ownership of riparian lands (Bronstein, 2002; Tewari, 2009) and subsequently replaced it with the concept of public trusteeship which is centred around the idea that no person may own water, but that the national government allocates, manages, uses and conserves water resources in the interest of the public (RSA, 1998). Consequently, the national government holds the authority to regulate the use, flow and control of water resources, which is executed by means of an administrative system for authorising the use of water. Section 22 of the NWA requires that all water uses be authorised in terms of either Schedule 1, a general authorisation, dispensing with the need for a licence, an existing lawful use, or a licence. Broadly speaking, the latter requires that a person wishing to apply for a licence needs to submit specified information to the responsible authority, which in turn needs to assess and review the information and furnish the applicant with a decision. Scholars have, however, argued that this prescribed process is substantively weak and does not include the necessary provisions to ensure that adequate information is submitted to the responsible authority for decision-making (King and Reddell, 2015). To address these shortcomings, the responsible authority promulgated the water use licence application and appeals regulations in 2017. The intent of the regulations is to provide a structured approach to the water use licence applications procedure and to provide details on the prescribed requirements for the processing of applications, site inspection meetings, public participation, the timeframes for receiving and processing of information in relation to an application and the contents of technical reports (RSA, 2017).

However, despite these regulations, the water use licence application process is still being plagued by procedural and substantive challenges. One such challenge relates to the quality of technical reports submitted to the responsible authority in support of water use licence applications. Anecdotal evidence seems to suggest that many of these technical reports are incomplete, contain weak impact assessments, lack thorough inputs by the public, generally do not comply with relevant guidelines, and in some instances are insufficient for decision-making purposes (CER, 2012; Myburgh, 2018; Pegasys Institute, 2018). In many cases, the general sub-standard quality of technical reports has led to notoriously lengthy delays in the processing and issuance of licences (Schreiner, 2013), which also undermines the ability of water use licence authorisations to achieve their intended objectives relating to human and environmental rights (Moolman et al., 2022).

As a means of evaluating practice, the reviewing of report quality and substance has been widely encouraged, especially within the fields of environmental impact assessment (EIA) (Hallatt et al., 2015; Swanepoel et al., 2019; Sandham et al., 2020; Alberts et al., 2022a; Malepe et al., 2022) and strategic environmental assessment (SEA) (Retief, 2006), and has contributed to capacity development efforts in these fields. Yet, to date, no effort has been made to empirically review the content of the technical reports submitted as part of the water use licence application process in an attempt to improve their quality. Therefore, it was the aim of this study to evaluate the quality, including the completeness and the substantive quality, of these technical reports in support of water use licence applications in South Africa.

The section below explains the research methodology adopted and implemented to achieve this aim. This is followed by the research results and discussion, and concludes with recommendations for improving the quality of submitted technical reports.

 

METHODOLOGY

The methodology followed 4 steps: (i) case study design, (ii) selection of the cases, (iii) data gathering and the development of the key performance indicators (KPIs) for evaluating the quality of technical reports; and (iv) evaluation process and analysis. The following sections discuss each of the 4 steps.

Case study design

In keeping with the aim of the research, a case study approach was adopted. Evaluation-based research suggests that case study approaches are appropriate (Alberts et al., 2022b). The case study design allowed the authors to predict similar results by using the same criteria within a similar context (Yin, 2018). The authors do not claim representation as for a representative sampling method, since the total number of technical reports submitted in support of water use licence applications in South Africa is unknown and, therefore, the paper adopted a multiple-case embedded design, i.e., multiple cases with multiple units of analysis (Yin, 2018).

Selection of cases

The selection of cases was informed by research undertaken by Flyvbjerg (2006), suggesting that an information-oriented strategy be adopted to maximise the utility of information from small samples and that the selection of cases be made based on expectations about the information content. When dealing with a case study approach to evaluation, the question of the number of selected cases needs to be addressed. The first point of departure is to take into consideration the phenomenon of saturation, whereby adding cases to the research might not necessarily be beneficial in contributing to the outcome of the evaluation and may have a negative impact on the quality of the research (Fusch and Ness, 2015). It should be noted that there is no ideal number of cases; however, a number between 4 and 10 may be sufficient (Eisenhardt, 1989). Therefore, this study opted to purposively select a total of 8 technical reports based on the following selection criteria:

• Technical reports submitted in support of water use licence applications between 1998 and 2021. This was to ensure that a wide range of technical reports may be obtained from the time of inception of the legal requirement for the reports in 1998 until the finalisation of the research in 2021

• All the selected technical report cases had to be associated with water use licence applications for section 21 water uses - 21 (c) impeding or diverting the flow of water in a watercourse, and 21 (i) altering the bed, banks, course or characteristics of a watercourse. These water uses are two of the most common water uses undertaken and are often applied for simultaneously. The undertaking of these water uses can have detrimental impacts on water quality and quantity, and therefore requires specialist investigation and recommendations. This corpus of information generated during the application process enabled us to create a detailed account of the phenomenon and patterns observed during the interpretation and analysis of results (Holloway, 1997)

• The water use licence authorisation was not, at the time of evaluation, subjected to any appeals process

The 8 purposively selected technical reports included water uses associated with large-scale mining operations (diamond and platinum mining), and smaller agricultural-related activities for the irrigation of crops and food production (poultry houses). The technical reports selected for evaluation were undertaken within various water management areas (WMA) and included the Lower Vaal WMA (now the Vaal-Orange WMA), Olifants WMA (now the Limpopo-Olifants WMA), Crocodile West-Marico WMA (now the Limpopo-Olifants WMA), Breede-Gouritz WMA (now the Breede-Olifants WMA ) and the Inkomati WMA (now the Inkomati-Ustuthu WMA). It should be noted that it is not the intent of the paper to compare the quality of technical reports across different time periods, economic sectors or different WMAs, but rather to provide an overview of report quality in general.

Data gathering and development of key performance indicators

Following the review quality approach developed for EIA by Norman Lee and Raymond Colley (Lee and Colley, 1990), which have been globally applied to review EIA report quality (for example, Lee and Dancey, 1993; McMahon, 1996; Nita et al., 2022; Sandham et al., 2020), a set of review criteria was developed which comprises an analytical framework. The review criteria, following Lee and Colley (1990), are divided into review areas (completeness and substantive content), which are further divided into categories, which are themselves divided into sub-categories. Completeness refers to whether the technical report covers all the required content, and substantive content refers to whether the information provided contributes to informed decision-making; this latter review area draws on the concept of substantive effectiveness, which is defined in research on EIA to be whether the objectives are met (Sadler, 1996). These review areas were supported by the selection of key performance areas (KPAs) for each review area and the development of related key performance indicators (KPIs). The KPAs are equivalent to the categories in the hierarchical review structure of the Lee and Colley (1990) EIA review package, with KPIs being equivalent to the sub-categories. To finally determine whether a KPA has performed well or not, we followed the guidance provided by Lee et al. (1999), who explain that the evaluation of the cases should start at the lowest level, in this case, the KPIs. Then, drawing upon this assessment, the evaluation moves upwards to the next level, in this case, the KPAs, until the overall evaluation of the cases has been completed.

The use of KPAs and KPIs is appropriate in evaluation research (Retief, 2006, 2007a, 2007b; Alberts et al., 2022b) and, for this paper, served to guide the authors in the evaluation process. Key performance areas selected were informed by existing literature (Retief, 2006, 2007b; Alberts et al., 2022a) and included for the review areas of completeness, documentation, determined water uses, technical information, description of the environment, key impacts and significance, mitigation and public participation.

For the review area of substantive quality, the KPAs included the description of the activity, socio-economic considerations, scoping, significance, mitigation and public participation. The set of KPIs developed was to ensure that a reasonable reflection of the quality of selected KPAs can be presented, and is by no means deemed to be a representation of all possible indicators. The developed KPIs were designed based on the following principles as suggested by Eckerson (2009), Jasch (2000), and Toor and Ongulana (2010):

• KPIs should be comparable between cases in order to indicate similarities and dissimilarities

• KPIs should be understandable and practicable and should be based on current and readily available information and data

• KPIs should be of a qualitative nature

• All KPIs should be developed based on the stated principles to ensure comparability

A total of 20 KPIs (12 KPIs related to completeness and 8 KPIs related to substantive quality) were designed based on the following:

• Relevant South African legislation regulating the water use licence application process

• Relevant South African guideline documents published providing guidance and a standardised approach to the application process

• Relevant peer-reviewed articles investigating the quality of environment-related reports

Whilst developed specifically for the South African context, these are potentially useful for the evaluation of technical reports associated with water licensing applications in any jurisdiction, subject to appropriate modification.

Evaluation process and analysis

The 8 purposively selected technical report cases were evaluated against the developed KPIs by 2 suitably qualified reviewers. Collectively, the reviewers had more than 40 years of experience in the fields of environmental management and law, water resource governance, and evaluation research. The evaluation process was confined to a review period between June and July 2022. The reviewers awarded an evaluation score to each of the KPIs by using symbols as defined in Table 1 below. The scores awarded for each of the KPIs were internally discussed amongst the reviewers before a consensus was reached, and the final score was allocated to the KPI. A summary of the evaluation results, review areas, KPAs and KPIs is provided in Table 2.

 

 

RESULTS AND DISCUSSION

The following section discusses the results of the 8 technical reports evaluated against the 20 KPIs developed. The results are structured around the 2 review areas, namely, completeness and substantive quality, and a discussion of each KPA per review area is provided.

Completeness

Figure 1 outlines the overall evaluation results related to the review area of completeness and associated KPAs.

KPA 1.1: Documentation

A total of 13 A's and 3 B's were scored for the 2 KPIs related to completeness of documentation over the 8 technical reports evaluated, indicating overall good performance. Section 41 of the NWA requires an applicant to apply for a WUL by using "specific forms" and containing "information as determined" by the responsible authority. A standard list of the general required information is provided in both the 'Internal and external guideline: generic water use authorisation application process' (DWAF, 2007a; 2007b), as well as in the 2017 'Water use licence application and appeals regulations' (RSA, 2017). Depending on the entity applying for a water use licence authorisation, the necessary registration documentation in support of the technical reports includes, but is not limited to, the proof of payment, registration document, trust certificate, letter of authorisation, black economic empowerment (BEE) certificate, the title deed, and all other relevant supplementary registration forms. The supplementary registration forms to be completed by the applicant depend on the type of water use applied for. For example, if an applicant applies for section 21 (c) and (i) water uses as an individual, the applicant is required to complete the following specific forms: DW756/769, DW763/775, and DW768/781 (RSA, 2017). The shortcomings in relation to KPI 1.1.1 were attributed to the applicants or companies not having a BEE certificate in place. Research by Myburgh (2018) suggests that these results are not unique. In a recent study where staff members from the responsible authority were questioned on the completeness of technical reports, 78% indicated that the submission of incomplete technical reports is a major factor influencing procedural efficacy of the application process (Myburgh, 2018). A research report published by the Pegasys Institute (2018) also noted that the submission of technical reports with incorrect registration forms leads to delays in the processing of applications and requires the responsible authority to reallocate vital resources to guide the applicant on the required information for a successful application.

KPA 1.2: Determined water uses

As indicated in the Methodology section the selected technical reports had to be associated with water use licence applications for section 21 (c) and (i) water uses, although they may have included additional water uses, for example, the taking of water from a water resource and storing of water. Evaluation of KPI 1.2.1 related to the inclusion of all determined water uses in the technical reports scored 6 A's, 1 B, and 1 C (see Table 2). Results showed that in the majority of cases all determined water uses were included in the technical report; however, in the one specific case that was scored a C, the technical report did not include the section 21 (c) and (i) water uses. These water uses were applied for after the submission of the technical report and were done so by only submitting the relevant supplementary registration forms. Analysis of the issued water use licence authorisation revealed that sections 21 (c) and (i) were indeed authorised. In the other case, which scored a B, all determined water uses were included in the relevant registration forms; however, they were not included in the technical report. This was apparently seen by the responsible authority as unnecessary and not a requirement for the completeness and submission of the technical report. Both these cases indicate inconsistent behaviour from the responsible authority and deviate from the guidance provided in the internal guideline document (DWAF, 2007a) and water use licence application and appeals regulations (RSA, 2017).

KPA 1.3: Technical information

For the 2 KPIs associated with technical information, 15 A's and 1 B were scored (Table 2). The technical information contained in the reports is required to ensure that the responsible authority can make an informed decision on the application submitted. Within the context of this paper, the nature of the technical information required is in the form of a wetland delineation report and the section 27 motivation statement. Appendix 6 of the water use licence application and appeals regulations sets out the content of the wetland delineation report and serves to provide the responsible authority with information on the boundaries of the wetland, including the type of wetland and associated soils, the ecological health, and the present ecological state of the wetland, including any potential impacts associated with the water uses applied for (RSA, 2017). The technical information required for the section 27 motivation statement is used to guide the responsible authority in the exercise of discretion to issue and to attach conditions to water use licences. In essence, the statement needs to contain information related to the need to redress past racial and gender discrimination and the socio-economic impacts of the proposed water use (DWAF, 2007a). All the evaluated technical reports included the necessary technical information; however, one technical report failed to include all the relevant aspects of the section 27 motivation statement. An explanation of the efficient and beneficial use of water in the public trust and a description of the socio-economic impacts of issuing or refusing the authorisation were omitted from the statement.

KPA 1.4: Description of the environment

The description of the environment in which the proposed activities and water uses will be located is crucial in justifying the determination of water uses and forms the basis for the assessment of risks related to the receiving environment, including water resources. The description of the environment (KPI 1.4.1) is in many cases supplemented by a map (KPI 1.4.2) which locates the proposed activity or activities, with the associated water uses applied for, at an appropriate scale (in most cases 1:50 000). Evaluation related to KPA 1.4 scored 10 A's, 5 B's, and 1 C. The description of the environment was of particular concern with KPI 1.4.1, scoring 2 A's, 5 B's, and 1 C. This was due to several of the technical reports failing to provide a description of the environment in relation to socio-economic factors, current land use, hydrology, and geology. Promising results were observed for the inclusion of a descriptive map of the location, and the proposed water uses (KPI 1.4.2), with 8 A's being scored.

KPA 1.5: Key impacts and significance

The determination of key impacts on the water resources is an essential step, since it informs the assessment of the significance of the risk and the subsequently proposed mitigation measures. In relation to KPA 1.5, which considered the inclusion of key impacts and significance, the technical reports scored a total of 10 A's, 5 B's, and 1 C. In the cases where weaknesses were observed, certain key impacts on the water resource were included in the wetland delineation reports, but did not make their way into the submitted technical reports. Similar results have been observed within the South African context. Wentzel et al. (2023) concluded that a lack of integration exists between information within specialist assessments and information submitted to the decision-maker. The determination of the significance of the impact associated with the water uses undertaken will ultimately determine the level of interventions needed to protect the water resource. However, once key impacts on the water resource have been omitted, one would not expect to observe any significance determination for such impacts. In one case, a complete list of key impacts on the water resource was provided; nevertheless, no evidence of the determination of significance related to the key impacts was included in the technical report.

KPA 1.6: Mitigation

The development of mitigation measures associated with key impacts and significance is essential to ensure that potential negative impacts on the receiving environment, including water resources, are avoided or minimised (Kidd et al., 2018). For KPA 1.6, a total of 6 A's and 2 B's were observed in dealing with the inclusion of mitigation measures for key impacts on the water resource. Interestingly, in some of the cases evaluated, mitigation measures included in the technical reports addressed certain key impacts, which were omitted from these reports (see KPA 1.5). It seems that in some instances, a disconnect exists between the determination of key impacts on the water resource, the significance of the impact, and the drafting of proposed mitigation measures. The authors are of the opinion that consultants responsible for the drafting of the technical reports tend to include generic mitigation measures for specific water uses and do not consider site-specific impacts. This poses a risk in that mitigation measures might be ineffective in mitigating the impact on the water resource.

KPA 1.7: Public participation

Involving the public in decision-making processes, such as water use licence authorisations, is essential to ensure the fulfilment of human and environmental rights and the achievement of sustainable development (Du Plessis, 2008). Section 41(4) of the NWA requires that the applicant inform interested persons and the public of the initiated application process. The responsible authority may require the applicant to give suitable notice in a newspaper by describing the licence applied for, including that written objections may be lodged against the application, an address where objections must be lodged, and other particulars as required by the responsible authority. Overall, good performance was observed for KPA 1.7 related to the inclusion of the public participation process in the technical report. Evaluation gave a total of 15 A's and 1 B. In one case, a statement for a planned public participation process was evident; however, the technical report contained insufficient evidence to verify whether the proposed engagement ever took place. These results are comparable with conclusions made by Alberts et al. (2022a), who found that public participation aspects within environmental impact assessment reports were well captured and addressed.

Substantive quality

The evaluation results for the substantive quality review area and associated KPAs are presented in Fig. 2.

KPA 2.1: Description of the activity

Good performance was observed for KPA 2.1, related to a satisfactory description of the proposed activity to inform the determination of all water uses. A total of 7 A's and 1 B were scored. These results can be attributed to the fact that final designs of the proposed development and activities were included in the technical report to inform the required water uses that needed to be included in the application for authorisation. Overall, these results are promising within the context of water use licence applications and contrast with other research studies investigating report quality. Wentzel et al. (2023) examined the quality of biodiversity impact assessment reports and concluded that the description of proposed activities was unsatisfactory and generally lacked details needed to inform decision-making.

KPA 2.2: Socio-economic considerations

Evaluation related to KPA 2.2 and the provision of sufficient information in consideration of issuing a water use licence authorisation as required by section 27 of the NWA, scored 5 A's, 2 B's and 1 C. The area of poor performance was related to the provision ofinadequate information on aspects including the socioeconomic impact of the proposed water use, redressing past racial and gender discrimination, efficient and beneficial use of water in the public interest and information related to the water resource, including the class of water and resource quality objectives. This is of concern, seeing that, although dated, guidance on this KPA is provided by the responsible authority in the 'External guideline: generic water use authorisation application process' (DWAF, 2007b). Various other instruments are also available to assist applicants in providing suitable information to inform decision-making. These instruments include the 'Water Allocation Reform Toolkit' (DWAF, 2007c) and the 'Assessment of Considerations for Water Use Applications v3.0.0' (DWAF, 2007a).

KPA 2.3: Scoping

Evaluation related to KPA 2.3 on the provision of sufficient information to identify key water-related issues (scoping) indicated good performance with 7 A's and 1 B. In one of the cases, the scoping of water-related issues was only included in the specialist assessment but excluded from the technical report. Overall, these results are encouraging, seeing that substantive information of this nature is crucial in justifying the identification of key water-related issues, which in turn form the basis of the risk assessment, thus enabling a focus on the most important issues to optimise the use of human and time resources. These results can also be attributed to the fact that the required specialist assessment associated with section 21 (c) and (i) water use applications is a wetland delineation study (RSA, 2017), which therefore essentially pre-empts the identification of key water-related impacts.

KPA 2.4: Significance

Mixed results were observed in the WULA technical reports for KPA 2.4, dealing with the use of a justified criteria and methodology for determining the significance of identified risks; 3 A's, 4 B's and 1 C were scored. These results may be attributed to the fact that 'significance' as a concept is not defined in either the NWA (DWA, 1998) or the water use licence application and appeals regulations (RSA, 2017), which therefore may lead to different interpretations of what 'significance' is, and the criteria and method to be used in determining significance. What has been noted in the majority of the evaluated technical reports and specialist assessments is the use of a quantitative risk methodology for the rating of subjective value judgements, which, in essence, is self-contradictory (Rossouw, 2003). This specific methodology (see RSA (2016) section 21 (c) and (i) water use risk assessment protocol) is the prescribed method for determining the various aspects of assessments related to the undertaking of (c) and (i) water uses and ultimately leaves room for the manipulation of borderline significance scores, which may be manually adapted. Another weakness related to KPA 2.4 is the fact that significance was determined only for impacts associated with the construction and operational phases of the proposed development and not the entire life cycle. These observations go against the 'cradle-to-grave' sustainable development principle of ensuring that responsible behaviour exists throughout the life cycle of projects (RSA, 1998). Finally, even though it is defined within the water use licence application and appeals regulations, it was evident that cumulative impacts in relation to water uses are poorly considered during the significance determination. These results are similar to observations made by scholars considering 'significance' and 'cumulative impacts' within the context of decision-making processes (Sandham et al., 2020; Alberts et al., 2022a). Based on the results, more guidance is required on how to deal with significance determination in technical reports submitted in support of water use licence applications.

KPA 2.5: Mitigation

Evaluation for KP 2.5 scored 2 A's, 4 B's, and 2 C's. Dealing with mitigation is considered to be closely related to significance determination (Alberts et al., 2022a), and since the entire life cycle of the proposed development and water uses were inadequately addressed (see KPA 2.4), it is to be expected that insufficient mitigation measures would be present in the technical report. Important aspects, including what should be done, by whom, by when, and what resources are required to implement the proposed mitigation measures, have not been defined or have been vaguely defined. In some instances, inconstancies were noted whereby the proposed mitigation measures by the specialists were never incorporated into the technical report submitted in support of the application. It is recognised that dealing with mitigation remains a challenge both within an international and South African context (Morrison-Saunders et al., 2004; Kidd et al., 2018).

KPA 2.6: Public participation

Aspects related to public participation were well addressed across all the cases, with KPA 2.6 being scored 19 A's, 3 B's, and 2 C's. The submission of a standalone public participation process (PPP) report containing a register of interested and affected parties, including a comments and response table, assisted in the verification of the developed KPIs. However, an area of concern within the PPP is the actual lack of participation by interested parties during the process, with certain cases having no or very few registered participants. Although it is recognised that attaining the participation of the public in environmental decision-making processes is difficult to achieve (Campbell and Im, 2016; Wentzel et al., 2023), it remains an international and national concern within the water sector (Dungumaro and Madulu, 2003; King and Reddell, 2015; Tsatsaros et al., 2018). Various guideline documents have been published by the responsible authority, which should be adopted and implemented by the applicant. These include the 'Generic public participation guidelines' (DWAF, 2001a) and 'A guide to stakeholder identification and involvement' (DWAF, 2001b). These guidelines aim at internalising and strengthening public participation within the water sector and attempt to assist in the identification and participation of key stakeholders. Much could be said about the substantive nature of the comments submitted in the cases, where interested and affected parties participated in the process, with many comments or questions seeking clarity on potential job creation and potential benefits to the surrounding communities, and not necessarily related to the water resource in question.

 

CONCLUSION AND RECOMMENDATIONS

The aim of this study was to evaluate the quality, including the completeness and substantive quality, of technical reports submitted in support of water use licence applications in South Africa. Evaluation results indicated that, overall, good performance was observed for most cases in terms of completeness (Review Area 1). This included documentation (KPA 1.1), inclusion of technical information (KPA 1.3), mitigation (KPA 1.6), and public participation (KPA 1.7). However, certain areas of concern were observed, including determined water uses (KPA 1.2), the description of the environment (KPA 1.4), and the determination of key impacts and significance (KPA 1.5) on the receiving water resource. In terms of substantive quality (Review Area 2), evaluation results showed areas of good performance related to the description of the activity (KPA 2.1), socio-economic considerations (KPA 2.2), scoping (KPA 2.3), and public participation (KPA 2.6). Areas of concern related to the substantive quality included significance (KPA 2.4) and the determination of mitigation measures (KPA 2.5). To improve the quality of technical reports, the following is recommended:

• Consistent practices: The concern around the determination of water uses (KPA 1.2) is not necessarily whether they have been determined or not, but rather the inconsistent practices regarding 'when' and 'how' the water uses are applied for and communicated. As indicated by the evaluation results, some water uses were only applied for after the submission of the technical report, and in other cases water uses were only included in the registration forms but not in the technical reports. These are examples where both the applicant and the responsible authority were not adhering to the procedural requirements of the water use licence application and appeals regulations (RSA, 2017) or any of the guidelines provided (see example DWAF, 2007a, 2007b). In essence, the recommendation is straightforward; both the applicant and the responsible authority must follow the prescribed regulations and guidance provided.

• Inclusion and integration of information: The omission of descriptions of important environmental features (KPA 1.4), such as the surrounding socio-economic factors, current land use, hydrology, and geology, is concerning and has a ripple effect on the quality of technical reports. It is recommended that this phase of information gathering be all-inclusive regarding what constitutes the environment (social, economic, and natural/physical) to ensure a seamless transition between the description of the environment and the determination of key impacts. It also seems that there is a lack of integration between the information provided by the specialist assessment (in this case, the wetland delineation report) and the technical report when it comes to key impacts and significance. From the evaluation results, it was evident that certain key impacts were identified and assessed (KPA 1.5) in the specialist report, yet were excluded from the submitted technical report. Reasons for this might be that the specialist assessment is seen as a standalone document without the need to integrate the observations made with the submitted technical report. However, several scholars have argued against this viewpoint (see Hallatt et al., 2015; Wentzel et al., 2023) and, in support of their concerns, we recommend an all-encompassing integration of specialist assessments and submitted technical reports.

• Need for guidance: From the results, it is evident that there is a need to engage with the concept of significance (KPA 2.4) to improve the substantive quality of the technical reports, especially when it comes to the methodology applied for significance determination. The authors are of the opinion that a qualitative methodology should be defined for the determination of what essentially is a subjective value judgement (Rossouw, 2003). It is reasonable to believe that if greater guidance and clarification on the concept of significance are provided from a methodological perspective, it will ultimately contribute to the improvement of other areas of concern within the technical reports, such as dealing with impact mitigation. Similarly, it is recommended that guideline documents be developed by the responsible authority which incorporate a holistic approach to the development ofproposed mitigation measures (KPA 2.4) and on dealing with the life cycle of the water uses undertaken. These guidelines should also provide information on mitigation measures for the cumulative impacts associated with the water use. Lessons can be taken from similar decision-making processes, such as environmental impact assessment and its published guideline documents dealing with mitigation and significance (see, for example, the then Department of Environmental Affairs and Tourism's Integrated Environmental Management Information Series Number 5: Impact Significance and Number 7: Cumulative Effects Assessment: DEAT, 2004a, 2004b)

• Meaningful engagement: Although KPA 2.6, related to public participation, performed well in terms ofthe developed KPIs, the authors would like to address the shortcomings observed within the process, seeing that involving the public and key stakeholders in decision-making processes, such as water use licence applications, is problematic, and an ongoing concern (King and Reddell, 2015; Tsatsaros et al., 2018). We also recognise that meaningful engagement with interested and affected parties is required to ensure the achievement of sustainable development objectives and the responsible management of water resources for future generations (Du Plessis, 2008). However, the lack of meaningful engagement on water-related issues observed within the technical reports is of particular concern to the authors. We encourage the adoption and implementation of existing guideline documents such as the 'Generic public participation guidelines' (DWAF, 2001a) and 'A guide to stakeholder identification and involvement' (DWAF, 2001b) in the effort to improve the substantive quality of the process.

The implementation of the proposed recommendations may assist in addressing the weaknesses associated with the completeness and substantive quality of technical reports. In so doing, the authors believe that good inputs (technical reports) will ultimately lead to better informed decision-making and good outputs (decisions regarding water use licences) (after Bond et al., 2017). We also trust that the paper serves as a step towards a discussion of an empirical nature on the quality of technical reports submitted in support of water use license applications in South Africa.

 

AUTHOR CONTRIBUTIONS

Jurie Moolman (corresponding author) conceptualised the research paper, drafted the initial outline of the manuscript, evaluated the case studies, and was involved in editing subsequent revisions. Reece Alberts evaluated the case studies, Francois Retief, Claudine Roos, Dirk Cilliers, and Alan Bond contributed to writing substantial parts of the paper, conducted analytical work, and contributed to the revision process.

 

FUNDING

This research was funded by the South African Water Research Commission under WRC Project No. 2022-2023-00750.

 

ORCIDS

Jurie Moolman: https://orcid.org/0000-0003-4848-5871

Reece Cronje Alberts: https://orcid.org/0000-0001-6840-4405

Francois Pieter Retief: https://orcid.org/0000-0001-7164-9593

Claudine Roos: https://orcid.org/0000-0002-6290-6129

Dirk Cilliers: https://orcid.org/0000-0001-9777-0463

Alan Bond: https://orcid.org/0000-0002-3809-5805

 

REFERENCES

ALBERTS RC, RETIEF FP, ARTS J, ROOS DP and FISCHER TB (2022a) EIA decision-making and administrative justice: the substance of just decisions. Impact Assess. Proj. Appraisal 40 (4) 296-304. https://doi.org/10.1080/14615517.2022.2066445        [ Links ]

ALBERTS RC, RETIEF FP, ROOS C, CILLIERS DP, MOOLMAN J, BOWERS J, MACGREGOR S, HENNMAN-WEIR F and OLIVIER I (2022b) Beyond legal compliance: the environmental performance of luxury safari lodges. Afr. J. Hospitality Tourism Leisure 11 (2) 710-726.         [ Links ]

BOND A, RETIEF F, CAVE B, FUNDINGSLAND TETLOW M, DUINKER PN, VERHEE R and BROWN AL (2017) A contribution to the conceptualisation of quality in impact assessment. Environ. Impact Assess. Rev. 68 49-58. https://doi.org/10.1016/j.eiar.2017.10.006        [ Links ]

BRONSTEIN V (2002) Drowning in the hole of the doughnut: Regulatory overbreadth, discretionary licensing, and the rule of law. S. Afr. Law J. 119 (3) 469-483.         [ Links ]

CAMPBELL JW and IM T (2016) Perceived public participation efficacy: The differential influence of public service motivation across organisational strata. Public Personnel Manage. 45 (3) 308-330. https://doi.org/10.1177/0091026016664899        [ Links ]

CER (Centre for Environmental Rights) (2012) Stop treading water: What civil society can do to get water governance in South Africa back on track. URL: https://cer.org.za/wp-content/uploads/2017/10/Stop-Treading-Water.pdf (Accessed 27 April 2020).         [ Links ]

DEAT (Department of Environmental Affairs and Tourism) (2004a) Overview of integrated environmental management, Information Series 5: impact significance. Government Printer, Pretoria.         [ Links ]

DEAT (Department of Environmental Affairs and Tourism) (2004b) Overview of integrated environmental management, Information Series 7: cumulative effects assessment. Government Printer, Pretoria.         [ Links ]

DU PLESSIS A (2008) Public participation, good environmental governance, and fulfilment of environmental rights. Potchefstroom Electron. Law J. 11 (2) 170-201. https://doi.org/10.17159/1727-3781/2008/v11i2a2762        [ Links ]

DUNGUMARO EW and MADULU NF (2003) Public participation in integrated water resource management: The case of Tanzania. Phys. Chem. Earth 28 (20-27). https://doi.org/10.1016/j.pce.2003.08.042        [ Links ]

DWAF (Department of Water Affairs and Forestry, South Africa) (2001a) Generic public participation guidelines. URL: https://www.dws.gov.za/Documents/Other/GPPG/guide.pdf (Accessed 12 April 2021).         [ Links ]

DWAF (Department of Water Affairs and Forestry, South Africa) (2001b) A guide to stakeholder identification and involvement. Water Quality Management Series: Managing the water quality effects of settlements. Edition 2, Policy Document U 1.3. URL: https://www.dws.gov.za/Projects/Dense/docs/How%20to%20guidelines/Stakehorlder%20Identification%20and%20Involvement.pdf (Accessed 1 August 2022).         [ Links ]

DWAF (Department ofWater Affairs and Forestry, South Africa) (2007a) Internal Guideline: Generic Water Use Authorisation Application Process. URL: https://www.environment.gov.za/sites/default/files/reports/externalguideline_genericwateruseauthorisation_applicationprocess.pdf (Accessed 18 February 2021).         [ Links ]

DWAF (Department of Water Affairs and Forestry, South Africa) (2007b) External Guideline: Generic Water Use Authorisation Application Process. URL: https://www.environment.gov.za/sites/default/files/reports/externalguideline_genericwateruseauthorisation_applicationprocess.pdf (Accessed 18 February 2021).         [ Links ]

DWAF (Department of Water Affairs and Forestry, South Africa) (2007c) A toolkit for Water Allocation Reform - A manual to help achieve race and gender equity in water allocation. URL: https://www.dws.gov.za/WAR/documents/WARToolkitJan07.pdf (Accessed 29 September 2025).         [ Links ]

ECKERSON WW (2009) Performance management strategies: How to create and deploy effective metrics. TDWI best practise report - First quarter report. URL: https://mindsight.com.br/wp-content/uploads/2020/08/How-to-Create-and-Deploy-Effective-Metrics-by-Weyne-Eckerson.pdf (Accessed 11 February 2021).         [ Links ]

EISENHARDT KM (1989) Building theories from case study research. Acad. Manage. Rev. 14 (4) 532-550. https://doi.org/10.2307/258557        [ Links ]

FLYVBJERG B (2006) Five misunderstandings about case-study research. Qualit. Inq. 12 (2) 219-245. https://doi.org/10.1177/1077800405284363        [ Links ]

FUSCH PI and NESS LR (2015) Are we there yet? Data saturation in qualitative research. Qualit. Rep. 20 (9) 1408-1416. https://doi.org/10.46743/2160-3715/2015.2281        [ Links ]

HALLATT TW, RETIEF FP and SANDHAM LA (2015) The quality of biodiversity inputs to EIA in areas with high biodiversity value -experience from the Cape Floristic Region, South Africa. J. Environ. Assess. Polic. Manage. 17 (3) 1550025. https://doi.org/10.1142/S1464333215500258        [ Links ]

HOLLOWAY I (1997) Basic Concepts for Qualitative Research. Blackwell Science, London.         [ Links ]

JASCH C (2000) Environmental performance evaluation and indicators. J. Clean. Production 8 (1) 79-88. https://doi.org/10.1016/S0959-6526(99)00235-8        [ Links ]

KARODIA H and WESTON D (2001) South Africa's new water policy and law. In: Abernethy CL (ed.) Intersectoral Management of River Basins. International Water Management Institute, Colombo, Sri Lanka.         [ Links ]

KIDD M, RETIEF F and ALBERTS R (2018) Integrated environmental assessment and management. In: Strydom H, King N and Retief F (eds.) Fuggle and Rabie's Environmental Management in South Africa. Cape Town, Juta.         [ Links ]

KING P and REDDELL C (2015) Public participation and water use rights. Potchefstroom Electron. Law J. 18 (4) 944-968. https://doi.org/10.4314/pelj.v18i4.06        [ Links ]

LEE N and COLLEY R (1990) Reviewing the quality of environmental statements. Occasional Paper Number 24. EIA Centre, University of Manchester, Manchester.         [ Links ]

LEE N, COLLEY R, BONDE J and SIMPSON J (1999) Reviewing the quality of environmental statements and environmental appraisals. Occasional Paper 55. EIA Centre, University of Manchester, Manchester. 72 pp.         [ Links ]

LEE N and DANCEY R (1993) The quality of environmental impact statements in Ireland and the UK. Proj. Appraisal 8 (1) 31-36. https://doi.org/10.1080/02688867.1993.9726883        [ Links ]

MACKAY HM, ROGERS KH and ROUX DJ (2003) Implementing the South African water policy: holding the vision while exploring an uncharted mountain. Water SA 29 (4) 353-358. https://doi.org/10.4314/wsa.v29i4.5039        [ Links ]

MALEPE KV, GONZÁLEZ A and RETIEF FP (2022) Evaluating the quality of Environmental Impact Assessment Reports (EIARs) for tourism developments in protected areas: The Kruger to Canyons Biosphere case study. Impact Assess. Proj. Appraisal 40 (5) 384-398. https://doi.org/10.1080/14615517.2022.2091055        [ Links ]

MCMAHON N (1996) Quality of environmental statements submitted in Northern Ireland in relation to the disposal of waste on land. Proj. Appraisal 11 (2) 85-94. https://doi.org/10.1080/02688867.1996.9727023        [ Links ]

MOOLMAN J, ALBERTS RC, ROOS C and RETIEF FP (2022) Identifying key risks to the effectiveness of water use authorisation systems through Theory of Change (ToC): The case of South Africa. Water 14 3830. https://doi.org/10.3390/w14233830        [ Links ]

MORRISON-SAUNDERS A, JENKINS B and BAILEY J (2004) EIA follow-up and adaptive management. In: Morrison-Saunders A and Arts J (eds.) Assessing Impact, Handbook of EIA and SEA Follow-Up. Earthscan, London.         [ Links ]

MYBURGH C (2018) Identification and critical analysis of the factors influencing procedural efficiency in water use licence application. Masters dissertation, North-West University.         [ Links ]

NITA A, HOSSU CA, MITINCU CG and IOJA IC (2022) A review of the quality of environmental impact statements with a focus on urban projects from Romania. Ecol. Inf. 70. https://doi.org/10.1016/j.ecoinf.2022.101723        [ Links ]

PEGASYS INSTITUTE (2018) Enhancing the water use authorisation framework: Simplified for small impact productive users. WRC Report No. 2536/1/17. Water Research Commission, Pretoria. URL: http://www.wrc.org.za/wp-content/uploads/mdocs/2536-1-17.pdf (Accessed 18 January 2020).         [ Links ]

RETIEF F (2006) The quality and effectiveness of Strategic Environmental Assessment (SEA) as a decision-aiding tool for national park expansion - the greater Addo Elephant National Park case study. Koedoe 49 (2) 103-122. https://doi.org/10.4102/koedoe.v49i2.119        [ Links ]

RETIEF F (2007a) A quality and effectiveness review protocol for strategic environmental assessment (SEA) in developing countries. J. Environ. Assess. Polic. Manage. 9 (4) 443-471. https://doi.org/10.1142/S1464333207002895        [ Links ]

RETIEF F (2007b) Quality and effectiveness of strategic environmental assessment (SEA) as a tool for water management within the South African context. Water SA 33 (2) 153-164. https://doi.org/10.4314/wsa.v33i2.49052        [ Links ]

ROSSOUW N (2003) A review of methods and generic criteria for determining impact significance. Afr. J. Environ. Assess. Manage. 6 44-61.         [ Links ]

RSA (Republic of South Africa) (1998) National Water Act No. 36 of 1998. Government Printer, Pretoria.         [ Links ]

RSA (Republic of South Africa) (1997) White Paper on a National Water Policy for South Africa. Government Printer, Pretoria.         [ Links ]

RSA (Republic of South Africa) (1998) National Environmental Management Act No 107 of 1998. Government Gazette 19519. Government Printer, Pretoria.         [ Links ]

RSA (Republic of South Africa) (2016) National Water Act, 1998 (Act 36 of 1998): General authorisation in terms of section 39 of the National        [ Links ]

Water Act, 1998 (Act No 36 of 1998) for water uses as defined in section 21 (c) or section 21 (i). Government Gazette 40229, 26 August 2016. Government Printer, Pretoria.         [ Links ]

RSA (Republic of South Africa) (2017) National Water Act, 1998 (Act 36 of 1998): Water use license application and appeals regulations. (GN R267). Government Gazette 40173, 24 March 2017. Government Printer, Pretoria.         [ Links ]

SADLER B (1996) Environmental Assessment in a changing world: Evaluating practice to improve performance - final report of the international study of the effectiveness of environmental assessment. International Association for Impact Assessment and the Canadian Environmental Assessment Agency, Ottawa. URL: https://unece.org/DAM/env/eia/documents/StudyEffectivenessEA.pdf (Accessed 10 December 2019).         [ Links ]

SANDHAM L, HUYSAMAN C, RETIEF F, MORRISON-SAUNDERS A, BOND A, POPE J and ALBERTS R (2020) Evaluating environmental impact assessment report quality in South African national parks. Koedoe 62 (1). https://doi.org/10.4102/koedoe.v62i1.1631        [ Links ]

SCHREINER B (2013) Viewpoint - Why has the South African National Water Act been so difficult to implement? Water Alt. 6 (2) 239-245.         [ Links ]

SWANEPOEL F, RETIEF F, BOND A, POPE J, MORRISON-SAUNDERS A, HAUPTFLEISCH M and FUNDINGSLAND M (2019) Explanations for the quality of biodiversity inputs to environmental impact assessment (EIA) in areas with high biodiversity value. J. Environ. Assess. Polic. Manage. 21 (2) 1950009. https://doi.org/10.1142/S1464333219500091        [ Links ]

TEWARI DD (2009) A detailed analysis of evolution of water rights in South Africa: An account of three and a half centuries from 1652 AD to present. Water SA 35 (5) 693-710. https://doi.org/10.4314/wsa.v35i5.49196        [ Links ]

TOOR SR and ONGULANA SO (2010) Beyond the "iron triangle": Stakeholder perception of key performance indicators (KPIs) for large-scale public sector development projects. Int. J. Project Manage. 28 228-236. https://doi.org/10.1016/j.ijproman.2009.05.005        [ Links ]

TSATSAROS JH, WELLMAN JL, BOHNET IC, BRODIE JE and VALENTINE P (2018) Indigenous water governance in Australia: Comparisons with the United States and Canada. Water 10 1-18. https://doi.org/10.3390/w10111639        [ Links ]

WENTZEL T, RETIEF FP, ALBERTS RC, MOOLMAN HJ, ROOS C and CILLIERS DP (2023) The quality of biodiversity inputs to environmental impact assessment (EIA) in the Succulent Karoo Biodiversity Hotspot, South Africa. J. Environ. Assess. Polic. Manage. 25 (4) 2350019. https://doi.org/10.1142/S1464333223500199        [ Links ]

YIN RK (2018) Case Study Research and Applications: Design and Methods. (6^th edn). Sage Publications, London.         [ Links ]

 

 

Correspondence:
Jurie Moolman
Email: jurie.moolman@nwu.ac.za

Received: 2 September 2024
Accepted: 22 September 2025