SPECIAL SECTION

 

The (im)possibility of monitoring and evaluating the impact of community engagement performance in South African universities: a review

 

 

V. O. Netshandama

Directorate Community Engagement, University Of Venda, Thohoyandou, South Africa. https://orcid.org/0000-0002-2466-6099

 

 

---

ABSTRACT

Monitoring and evaluation (M&E) allow universities to learn from past experiences, improve service delivery, plan, and allocate resources and demonstrate results as part of their accountability to stakeholders. M&E also assists in keeping projects on track, providing a basis for reassessing priorities and creating evidence-based data for projects. Considerable scholarship is illuminating insights and sharing experiences of community engagement in higher education. However, the question of M&E remains contestable due to the complexities in implementing community engagement in higher education. This article discusses the (im)possibility of facilitating M&E in higher education community engagement spaces. In 2009, the University of Venda established a directorate for community engagement, which I became its first director. I use the insider lens to reflect on the intersections of concepts and constructs such as knowledge coproduction, ecosystems, societal impact, and an M&E framework in higher education. I intend here to bring some problems and contestations in generating a framework for monitoring and reviewing community engagement, arguing that the framework should embrace multiple ontologies and be intentional about robust engagement with epistemological, ontological, and ethical questions around exclusivity and dominance. Hopefully I will add a much-needed dimension acknowledging the complexity of a community engagement discourse - toward an inclusive, participatory-ecosystemic way of self-assessment, monitoring and evaluation.

Keywords: monitoring and evaluation, community engagement, South African Higher Education, Societal impact, logic framework, community engagement programmes, higher education, ethics

---

 

 

CONTEXTUALISING THE PROBLEMATIC SPACES OF COMMUNITY ENGAGEMENT AND MONITORING AND EVALUATION IN SOUTH AFRICAN UNIVERSITIES

The two constructs of (integrated) community engagement and monitoring and evaluation are problematic on university grounds. The problems are not unique to South African universities. By the 1990s, academic leaders on both sides of the Atlantic discussed the insufficiency of the standard model. In the USA, Ernest Boyer's seminal paper, "Scholarship Reconsidered", proposed that the dominant view of scholarship - disciplinary research assessed by peer-reviewed publications - did not adequately describe the many functions academics need to perform in the modern university: from teaching to interdisciplinary research, to engagement with society's problems.

"At one level, the scholarship of engagement means connecting the university's rich resources to our most pressing social, civic, and ethical problems. Both students and professors would view campuses not as isolated islands but as staging grounds for action. But, at a deeper level, I have this growing conviction that what's also needed is not just more programs but a larger purpose, a larger sense of mission, and a more considerable clarity of direction in the nation's life as we move toward century twenty-one. Increasingly, I'm convinced that, ultimately, the scholarship of engagement also means creating a unique climate in which the academic and civic cultures communicate more continuously and creatively with each other, enriching the quality of life for all of us." (Boyer 1996a, 19-20).

This quotation assists the framing of the critical discussions in this article. In addition, Boyer's subsequent paper on from scholarship reconsidered to scholarship assessed (1996b, 129-139), and (Cooper 2011, 344) seminal work about "The University in Development" assist the framing of this article. Cooper explored challenges associated with pursuing a new "third" mission critiquing the "orphan" status of the fourth helix in the university-community relations. Creating a unique climate where academic and civic cultures communicate more continuously and creatively with each other is more of a utopia than a norm (Boyer 1996a, 23; Boyer 1996b, 129-139; Cooper 2011, 344). Universities and the State did not seem to take community engagement seriously in South Africa. Community engagement in Higher education remained the distant other until recently, two decades later since the 2006 Bantry Bay conference.

The foundational challenge I put forward is that to reflect on the possible framework for M&E, is to navigate the spaces in between the normative of cognitive capitalism, and the freedoms to imagine a critical post humanist field of study that acknowledges ecologies of knowledges engaging in reciprocally with mutual respect (Braidotti 2019, 31-61). Often canvassed as an integral part of the scholarship of teaching and learning and research, I first acknowledge the "borrowing" of concepts and construct from across disciplines and broad fields of study such as Biology and Environmental Sciences. This borrowing is reflective of the very nature of community engagement, which lends itself in interdisciplinary spaces. I also wish to introduce the first (im)possibility, which is the absence of norms and standards in community engagement spaces. This is compounded by the complexities associated with the integrated nature of community engagement and the scholarship of engagement. The current CHE institutional review guide explains:

"Integrated community engagement as one of the core functions of higher education involves working constructively and co-operatively with communities which are connected to the institution, in order to make that institution more adaptive and responsive to needs that it could service. Such integrated community engagement has the potential to affect or influence almost every aspect of an institution's functioning. Community engagement should be specifically integrated with learning, teaching and research and should be based on and enhance the disciplinary knowledge and expertise of the institution." (Council on Higher Education 2021, 7).

Universities have grappled with community engagement manifestations to date. A consensus in the understanding that South African universities exist for teaching and learning, research, and community engagement had been reached as the White Paper 3 (DoE 1997) was being interpreted and applied. The Higher Education Amendment Act (DoE 1998), and the National Plan for Higher Education (DoE 2001), also indirectly assisted the sensitisation towards intentional community engagement to bring about sustainability and transformation in South African Higher education.

The statement on the Higher Education Quality Control (HEQC) placed community engagement as an integral part of teaching and learning (HEQC founding document 2001). Research provided some understanding, albeit with some anxieties, about applicability (Bender 2008, 87-89; Bhagwan 2017, 171-185; Mbah, Johnson, and Chipindi 2021, 1-7). Later, the NRF community engagement funding stream and the vast array of publications that sought to conceptualise community engagement and community engagement models got the sector to a point where we could start to reflect on our M&E (mal)practices (Bender 2008; Bhagwan 2017).

Key events took place that are attributable to the presence of community engagement in South Africa are the 2006 Bantry Bay conference, and the 2010 Department of Science and Technology's Community University Partnership Programme (CUPP) drive, which drew inspiration from Brighton University in the UK (Hart and Northmore 2011, 34-58). CUPP targeted the active participation of at least five rural-based universities between 2010 and 2012.

I believe the 2006 Bantry Bay conference launched the discursive conceptual phase of community engagement. At that point, there was difficulty in mainstreaming community engagement as a knowledge field without a theoretical-conceptual understanding of what it entails. At the conference, Michael Gibbons (2006) argued that there should be a distinction between universities that have engagement as a core value and those that do not. He addressed the notion of a prevailing social contract between the university and the community in a "mode two" knowledge production and the need for a larger view of knowledge production than that of the standard university.

Limoges et al. (1994, 3) clarified the difference between Mode 1 and Mode 2 knowledge production, which Gibbons referred to in his presentation at the conference.

"In Mode 1 knowledge production, problems are set and involved in a context governed by the largely academic interests of a specific community. By contrast. Mode 2 knowledge is carried out in a context of application. Mode 1 is disciplinary while Mode 2 is transdisciplinary. Mode 1 is characterised by homogeneity. Mode 2 by heterogeneity. Mode 1 is hierarchical and tends to preserve its form. while Mode 2 is more heterarchical and transient. Each employs a different type of quality control. In comparison with Mode 1. Mode 2 is more socially accountable and reflexive."

Limoges et al. (1994, 3), I would suggest, recognised the need for a differentiated M&E then. Similarly, Boyer (1996b, 129-139), Schön (1995, 27-34) and Gibbons (2006, 24) and others asserted the need for new epistemologies. An "agora", according to Gibbons (2006, 24) is a transaction space - a melting pot of all kinds of discussions about knowledge. The agora refers collectively to the public space in which "science meets the public", and in which the public "speaks back" to science. Not necessarily a new physical location, but a common name for places, networks, and institutions with influence on socially robust knowledge. These spaces, I argue, have been difficult to locate in universities to date, which probably could be associated with the obscurity of the practices of M&E. At the centre of universities, the core mandate is the knowledge production project. The postcolonial indigenous methodology frameworks and indigenous relational methodologies advance collaborative research that includes communities' voices and thus suggest notions of co-production of knowledges instead (Chilisa 2017, 813827; Chilisa, Major, and Khudu-Petersen 2017, 327). However, these methodologies are still received with resistance on university grounds in part due to persistent dominance and privileges of one knowledge form over others (Braidotti 2019). So, the second (im)possibility, I argue, relates to the discomfort associated with the argument that the M&E, generally understood, would be skewed to a dominant paradigm, defeating the very idea that community engagement would be integral to the values of knowledge ecologies. Gibbons (2006, 3-5) states,

(i) "a new social contract between society and science is emerging;

(ii) it will be constructed upon the opening up of the universities to the contextualisation of research, their participation in the agora, and their involvement in the production of socially robust knowledge; and that

(iii) these elements can provide a framework within which to ascertain whether or not individual institutions have embraced engagement as a core value." (Gibbons 2006).

The launch of the South African Higher Education Community Engagement Forum (SAHECEF), http://www.sahecef.ac.za/ on 2-3 Nov 2009, is perhaps also one of the catalytic forces behind the staying power of community engagement in South African universities. In addition, are collaborations with, and presence of international organisations such as the Talloires network (https://talloiresnetwork.tufts.edu/who-we-are/), the Global University Network for Innovation (GUNI) (https://www.guninetwork.org/), CLAYSS - Centro Latinoamericano de Aprendizaje y Servicio Solidario (www.clayss.org), the UNESCO Chair in Community Based Research and Social Responsibility in Higher Education Based at the University of Victoria (UVic), and the Society for Participatory Research in Asia (PRIA) (https://www.unescochair-cbrsr.org/). Furthermore, the National Research Foundation (NRF) made possible several lectures by colleagues from USA, Australia, Canada, and others.

The third (im)possibility has to do with the sufficiently canvassed complexities of defining a community. For over a decade, contestations in the community engagement space in South Africa have centred around the slow progression towards discussions about societal impact and the evaluation thereof. There are multiple definitions of the term "community", which indicates its highly contested nature (Banks and Butcher 2013, 13-15; (Nkoana and Dichaba 2017, 177196). Nkoana and Dichaba (2017, 179-182) group the definitions into five broad categories: descriptive communities, territorial communities, communities of interest or identity communities, normative communities, and active communities. Reference is often made to "community leaders" and "community stakeholders", consultations, and community participation, even as these add to the complexities of the phenomenon of community engagement and, therefore, the M&E of community engagement. Who is a community leader? Who is a community stakeholder? What sort of consultation is sufficient? How (im)possible is it to apply the principle of inclusivity in participation? These questions will continue to justifiably un-settle the space. In this regard, the argument of contextualisation and programme/project level evaluation remains the critical foundational step. A further question could thus be asked: can a cumulation of programme level evaluation sufficiently provide an institutional and hence a country picture?

I adopt the concept of an active community in this article. This type of community builds on and encompasses the descriptive, territorial, identity, and value (normative community) meanings identified above. It refers to collective action by members of territorial or interest communities that embraces one or more of the communitarian (Etzioni 1995a; 1995b) values of coherence, participation, solidarity (Tapia 2012, 187-203), and Ubuntu (Nkoana and Dichaba 2016, 213-224) This is the idea of community that decision-makers often have when they seek to promote initiatives that draw upon community strengths and capacities (Banks and Butcher 2013,13-15). In the context of this article, I utilise the concept of active communities to refer to external stakeholders that collaborate with university staff members in university-community engagement endeavours. The term "active" also implies that external stakeholders are not mere spectators and passive recipients of "benefits" from universities but are actively involved and work alongside university staff members in university-community engagement efforts, including being actively involved in monitoring and evaluation processes. This would help to strengthen the practice of community engagement in higher education. Engagement is when two or more partners, such as people, organisations, and nations, enter into a symbiotic agreement (i.e., development cooperation agreement) or mutual destruction (such as in armed conflicts). The essence of engagement is that both parties actively participate. Since there is an expectation of participation in action, it goes without saying that the monitoring would inevitably be inclusive of participation principles (Bhagwan 2017, 171-185).

In the case of engaged teaching, learning, and research, philosophers, have challenged an individualistic approach to learning where the emphasis is on the individual per se, rather than on the individual in context and relationships, including relations with all living beings and the environment. Community engagement programmes provide opportunities for learning about learning and questioning the notions of exclusive normative universalism (Habermas 1995, 111 -115). I argue that when students and communities participate in authentic community engagement programmes, learning becomes a humbling experience, rather than an arrogant act of showcasing what is already known, which would defeat the very idea of why we should engage in the first place. Community engaged teaching and learning requires energy, courage, hard work, patience and is somewhat messy. Everything happens simultaneously, learning, unlearning, inquiry, knowledge co-production, sharing, exchange, caring, etc. An inquiry is part of a learning and of caring and development, you do not do one on Monday and the other on Sunday. M&E framework of community engagement in higher education should acknowledge that the quality and impact of community cannot be understood with the same framework that perpetuated categorisation and privileges of one (research) over the other (teaching and learning) or vice versa.

Like countries such as the USA, Europe, Latin America, and others, service learning has gained traction in SA. Some universities have successfully infused compulsory service-learning modules or courses, and there have been good practice guides from CHE in this. However, there are some discomforts regarding the extent to which those modules are evaluated and are influencing the pace at which learning, and teaching may be transformed. I argue that service learning may not necessarily translate to community engagement co-creation principles, even if it looks good on paper. Notions of service learning are limited mainly by worldviews of dominance over another form of learning. Effective learning occurs when there is an element of doubt about what is considered the truth. When there is an inclination to question that truth and to embrace other truths, then learning will occur in perpetuity, yet evaluative methods seem to be limited to the extent to which they can embrace co-learning and teaching. This is emphasized by several scholars (Keane, Khupe, and Seehawer 2017; Khupe and Keane 2017), who insist that the role of universities as transmitters of culture, learning and independent thought should consistently be questioned. Similarly, communities do have critical attitude, if not suppressed, they can actively participate in the monitoring and evaluation of community engagement.

Brink (2018, xvii) argues that the growing challenge to the standard model was reversed by the rise of international rankings, their impartial nature, "relentless linearity", and their negative impact on society, for example, the listing of the top 500 universities in the world by Shanghai University in China, and the Times Higher Education ranking. Today's incentives encourage the wealthy and well-connected to game the system, passing on their privileges to their children in ways that exacerbate economic and social inequality. Brink's (2018, xvi) popular argument is that universities need a focus on what they are "good for" in addressing society's problems, in addition to the conventional assessments of what they are "good at", i.e., disciplinary research. In doing so, universities will inevitably interrogate a larger purpose in determining their suitability.

Co-production is "the bringing together of different knowledge sources, experiences and working practices from different disciplines, sectors, and actors to jointly develop new and combined knowledge for addressing societal problems of shared concern and interest" (Visman et al. 2022, 3). Despite increased consensus about community engagement as an integral part of learning, teaching, and research. To co-produce as a route to developing impactful work, defining metrics for monitoring, and evaluating the impact of co-production in community engagement programmes of universities remains a less attractive task due to its complexity and impossibilities. In community engagement projects, knowledge production occurs in the "agora" (Gibbons 2006, 12), a transaction space par excellence that produce socially robust knowledge. Knowledge co-production occurs in with communities. Power is redistributed through negotiation between community members and researchers or scientists. Planning and decision-making responsibilities are shared. The poor and powerless citizens can negotiate and engage in trade-offs with power holders, for example, through joint committees and reflection in action (Cornwall 2008, 269-283; Schön 1995, 27-34).

Community engagement could offer possibilities to disrupt the dominance of certain parties. However, it is a highly contested space. The fundamental principle of community engagement is reciprocity. Reciprocity, in the true sense, suggests a significant amount of listening rather than talking on the part of university researchers and the university community in general.

Part of the protestations are due to the inherent desire to maintain the status quo. University community engagement facilitation is still skewed towards positivist domination. Whereas scholars in participatory fields argue that every practice of knowledge production is relational. We can barely translate that through our engagement with students, let alone communities external to us. Meetings about knowledge co-creation processes are often called in terms of universities' prescripts and limited understanding of what a relatable practice of knowledge production is. The agenda is often predetermined with little room for manoeuvre, and there is usually no trace of evidence regarding how community consultation and feedback data are integrating a university's systems, policies and processes. Universities continue to be ivory towers and mirror the already existing "us and them" psychology in the communities. Community engagement could offer possibilities to disrupt the dominance, as mentioned. However, it is a highly contested space.

I use monitoring and review as a self-reflective (Schön 1995, 27-34) process because it is indeed a space that allows us holistic approach to learning and living yet it is flooded with templates that are suggestive of tick box syndrome, power, policing, and protectionism. Evaluation methods, in the main, continue to reflect a Euro-Western perspective (Bamberger 1991); a view that may not necessarily advance our understanding of diverse cultural living, located learning and experiences. According to (Bhola 2003, 398),

"The process should serve not only to induce self-awareness and reflexivity, and thereby bring clarity, coherence, and commitment to one's own standpoint, but also should compel similar reflection on the part of "others" (scholars and stakeholders), helping to create "an inter-subjective space in which difference can unfold in its particularity", which in turn may lead to enlightened agreement and honest disagreements."

Some evaluators and researchers have applied participatory evaluation methods (Chouinard and Cousins 2015, 5-39). Participatory evaluation is best considered an umbrella term under which various approaches can be classified (Cullen and Coryn 2011, 32-47). Whilst the application of the constructs of (integrated) community engagement and monitoring and evaluation may differ per continent, they are all primarily intended to disrupt elitism; notions of a knowledge production process that is exclusive and is a continual expression of dominance, the unsustainable culture of knowledge for knowledge's sake, and singularity of knowledge forms that South Africa cannot afford.

On a positive note, the Council for Higher Education (CHE) has recently adopted the self-evaluation process in institutional and programme reviews (CHE 2004); these include a self-assessment as a first step, which I believe gets us to reflect as accountable citizens of the universe (Braidotti 2019). Universities have for centuries regarded themselves as the holders of dominant knowledge paradigms. Those challenged used the same strategy that created the hegemonic normative Foucault 1972 and 1977 (as cited in Gordon and Grant 2004). As a result, marginalised communities did not necessarily have a voice. The dominance meant that the much-needed review of impact was skewed towards the dominant paradigm and the funding influence. This is dangerous as it perpetuates elitism and disenfranchises the marginalised communities (Chilisa et al. 2017, 326-329; Vogel, Steynor, and Manyuchi 2019, 3-8; Vincent et al. 2018, 48-58. According to Connell (2016), the power of "coloniality" has maintained the continuation of hegemonic forms of knowledge in the global economy. However, it is worth noting that despite the colonial dominance, there is a growing force that recognises that there can be no relevance and impactful work of the university that plays and referees itself without acknowledging the real referees and spectators. Therefore, I argue that the proposed M&E frame should apply the principle of restorative justice by insisting on active participation of communities who should be fans that admire the value-add universities make to its context, first and foremost.

 

THE CASE FOR AND THE (IM)POSSIBILITY OF MONITORING AND REVIEW OF AN INTEGRATED COMMUNITY ENGAGEMENT FRAMEWORK

I use the definition of M&E as explained by Crawford and Bryce (2003, 363-373) to mean the tracking of project outputs and outcomes as indicators of project effectiveness or the extent to which the project achieves its stated objectives. This definition hopefully elevates the significance of locating M&E of community engagement projects within the intentions and context in which they would have been established. There is a growing interest in measuring social impact across organisations, businesses, government, and non-profit sectors. Universities are no exceptions. The term "social impact" is a complex construct with its origin in the social enterprise sector (Ebrahim and Rangan 2014, 118-141). I borrow from the developmental programme evaluation (Chen 2006; Patton 1994) to refer to social impact as "a logic chain of results in which organisational inputs and activities lead to a series of outputs, outcomes and ultimately to a set of societal impacts" (Ebrahim and Rangan, 2014, 118-141).

The fourth area of (im)possibility relates to the slower pace in which key constructs, social impact and "monitoring and evaluation" are developing and actively utilised on university grounds and the difficulties in linking activities to impact because of difficulties with attribution and causality questions (Maas and Liket 2011, 177). It has been conceptually harder to develop categories and measurement methods of social impact, also because of inherent necessity to delineate short- and long-term effects to society and the nexus between economic, environmental. However, I would argue that in recognising the role that integrated community engagement monitoring in universities may play in enhancing individual and societal impact, there is a case to be made for the university to provide a framework for infusing societal implications in the matrix for measuring how universities are performing. Such a framework could be guided by the theory of change (TOC) and the logic model which is a useful one for M&E. It is largely utilised in non-profits organisations to measure impacts against investments. However, the framework is now applied across sectors. The discussions about the third mission of universities include program logic that depicts the cause-and-effect relationships between program activities, processes, and outputs as well as short, intermediate and longer-term outcomes. The starting point should be self-assessment, i.e., assessment of the (social) impact that one has on the roles, ascribed, or achieved - beyond counting who else is reading one's work. Furthermore, the work done through the following are acknowledged as useful in these discussions:

i) the global rankings and global reporting framework, for example, the Global Reporting Initiative (GRI), which sets out the principles and indicators that organisations use to measure and report their economic, environmental, and social performance, and

ii) the Most Significant Change (MSC) framework, which involves the collection of significant change stories from people engaged in programs, and the systematic selection of the most important of these stories by selected panels.

These, I suggest can be used for benchmarking, however it should be noted that each framework and method have its benefits and limitations. Therefore, I want to concur with the notion that it is strategic for institutions to first focus on clarifying the purpose of the monitoring and evaluation. Furthermore, I borrow from Maughan (2012, 19-21), who suggests a process that can aid in developing an M&E framework to include:

i) starting with the self,

ii) placing the theory of change as foundational to M&E scope,

iii) ensuring that each individual contributor reflect on and share personal impact statement,

iv) leveraging on the strength (what is already there-community engagement projects database), location, and context,

v) clarifying the "good for" logic, what the institution is looking to be great at,

vi) setting the measures and evidence of success,

vii) developing a meaningful inclusive monitoring system,

viii) developing a strategic evaluation system, and

ix) incorporating self-reflections, co-learning, and improvement strategies.

x) ensuring effective reporting.

Engagement of any form implies holding each other accountable. It provides spaces for community members to ask questions, including the difficult ones to which universities may not have answers to. When communities formulate questions, they would not necessarily follow the prescripts of a discipline. They present the problem as they see it unfold, not according to terms and sabbaticals. There is a large amount of arrogance in the knowledge production space. Universities are quick to remind communities how they function, and not vice versa. However, (Odora Hoppers 2002, 2-20) points out that a movement and network are necessary, which should not only deliberate but develop innovative dialogic programmes to engage the world meaningfully.

There seems to be a consensus amongst universities that the logic framework could be used to encourage the development of standard measures and indicators at each level of contribution (inputs, activities, outputs, outcomes, and impacts) for each of the following four categories: exerting influence and promoting change, connecting the community, and sustainability. Such would start at the project or programme level by applying an ecosystem review approach (Galan-Muros and Davey 2019; Markiewicz and Patrick 2015).

 

SO HOW SHOULD WE MONITOR AND REVIEW FOR IMPACT?

We are in a great space as a county in that we seem to be aware that with the parallel transformation and decolonial discourse in higher education, the M&E will have to take into cognisance the generally accepted values of social justice in education, cognitive justice, inclusivity, and transparency. These put voices of marginalized groups in the communities in the same discursive space with an educator-researcher as a collaborative leaner. Communities should contribute to how the impact should be monitored and pronounced. I propose the eco-systemic approach to monitoring and review. The ecosystem concept has its roots in biology but was later transferred to management and other disciplines. An ecosystem can generally be described as a set of parties that interact while being mutually dependent on each other's capabilities to achieve a shared value proposition (Adner 2017; Adner and Kapoor 2010; Lusch and Nambisan 2015).

Enabling sustainable systems for participatory monitoring and review can bolster institutional capacities to demonstrate the value of community engagement. Through identifying continuous strategies for community feedback and strengthening capacities and practices of reflexivity in community engagement, South African universities would have an additional space to engage in the troubled past and present, as well as the uncertain futures of the Higher Education sectors. The emphasis on co-learning has value in recognizing the importance of ongoing dialogue and feedback, not only in meeting donor reporting requirements and supporting project management, but also in maximising the impact of the coproduction process while informing research. Moreover, monitoring the steps in the coproduction process enables tracking of incremental change in advance of more fundamental or transformational change to which community engagement initiatives can contribute.

Participatory monitoring and review approaches that see no disciplinary boundaries, that actively involve the project stakeholders, and particularly those people most directly impacted by an issue, can strengthen ownership and sustainability (Vincent et al. 2018, 48-58; Visman et al. 2022, 1-17). Levels at which monitoring, and review should occur, therefore, are in project and or programme planning, divisional and strategic levels, tracking the input, reflecting on the processes, and counting the numbers, as well as digging into the details of the change/s that took place and the accompanying transformation. In this regard, however, there is another area of (im)possibility when project "owners" sometimes do not see the need to report on these institutionally for various reasons, including the sentiments that those who do community engagement work do so mainly from their unsupported, externally funded projects or programmes, and thus feel more liable to the funder.

I am aware, however, of the difficulties in proving evidence that the change has occurred because of a programme-level input (Hart and Northmore 2011, 34-58). However, possibilities exist that as researchers and M&E practitioners continue to interrogate, design frameworks and tests, there will be a simultaneous reduction of existing gaps, even though this may mean creating new ones to allow contextualisations and thriving M&E ecosystems.

I argue that as a problematic policing notion of M&E disappears, reflexivity, which asks questions such as, "am I responding to anything of value to someone other than myself?", and reflectivity, which asks questions such as, "am I checking my practices regularly?", and "do I allow collaborative reflections with communities?" will be an ideal replacement. When community engagement is an integrated and integral phenomenon, we will peer review one another effectively. We would be quite inviting and humble to the reality that our current systems have much to offer yet they can also still be improved for relevance and societal impact. To do this I propose a M&E framework and approaches for monitoring and reflecting on the input, processes, and outcomes of integrated community engagement programme investments be deliberately linked with the social impact role of universities, which inevitably would compel programme owners to include the voices of people affected from planning the project to reviews and to post programme evaluation. This framework would combine values principles- and process-based approaches to track changes amongst the various parties involved in co-producing knowledge and impact, including previously marginalised communities.

However, it should be noted that the proposed development and application of this framework raises fundamental questions regarding the metrics for measuring the effects of coproduction and the principles - including inclusivity, diversity, and value for all partners in the process - on which these are based. Capacities must be built from the initial framing, baselining, ongoing monitoring, knowledge management and learning, mid-term and final evaluation. This framework may contribute to the emerging field of M&E of community engagement coproduction and may help improve the field's robustness going forward. In addition, there is a case to be made that whilst funders increasingly require ethical standards to be upheld, there is growing literature pointing to the sensibilities around the ethics in knowledge co-production e.g., (Braidiotti 2019; Greenaway et al. 2022; Hagendijk 2015; West 2022). Existing ethics standards and principles provide an essential foundation for multi-partner community engagement initiatives.

 

A PROPOSAL TO EMBRACE KNOWLEDGE CO-PRODUCTION PRINCIPLES IN THE M&E GOALS

The pioneering work of Paolo Freire, Karl Marx, Antonio Gramsci and Jurgen Habermas, False-Borda (1987, 329-347) and others embraced concepts of emancipation, liberation, inclusivity, and others designed to ensure that the least powerful would play a crucial part in the knowledge creation process, influenced my proposal (Chouinard and Cousins 2015, 6). Steven Kemmis's presentation at the collaborative webinar of multiple universities from transdisciplinary perspectives (Webinar presentation, 29 June 2022) entitled "Participatory action research: methodologies for transformation" quoted Karl Marx in the eleventh of the Theses on Feuerbach: "Philosophers have hitherto only interpreted the world in various ways; the point is to change it." Kemmis (2022) applied this quote generally to point out that the aim of education should be transformation. I argue here that the aim of M&E of community engagement in Higher education should be to transform it towards the epistemic justice. In this regard, further discussions could be made about integrated community engagement, and the M&E thereof as a space and a tool for transformation in higher education, expanding the societal impact and relevance agenda, which, until recently, has not been emphasised in higher education in SA (Kemmis et al. 2014, 25-41).

The aims of M&E shape the appropriate approach, for example by indicating relevant stakeholders and the balance between timeliness and comprehensive data gives one a sense of what change needs to happen and how that change may be explained with evidence. The first step in designing an institutional M&E framework is to confirm with stakeholders the purpose of M&E. In this regard, the purpose of M&E is to strengthen the effectiveness of community engagement in higher education and to learn about gaps and areas for improvement from an inclusive lens. The M&E design, therefore, should be inclusive, participative, consultative, and as reciprocal as possible, particularly bringing in the voices of the previously marginalised. Community engagement implies holding each other accountable. In infusing reciprocity as part of the M&E principle, the inherent social contract is accordingly carried to monitoring quality and societal impact.

I believe that implicit in the design should be a deliberate attempt at developing and aligning change theories. Universities have made several strides in this regard, for example, some universities have initiated calls for community engagement proposals. These calls allow them to be facilitative and build a critical mass capacity. The funding calls of NRF and other international collaboratives and partnership calls, such as the UNDP and the British Council, which further require researchers to indicate a theory of change to be employed in their projects are assisting the shift towards intentionality and inclusivity. Similarly, there are visible efforts of funders to embrace participatory action research theories, designs, and methodologies. This allows the researchers to reflect on parameters such as SDGs, agenda 2063, NDP, and others during project planning. Furthermore, it allows universities to reflect and indicate their contributions to these. To satisfy the conditions, researchers commit to reporting on the progress made regarding impactful activities by talking to these indicators. Therefore, the improvement should ask to what extent these reflections include community stakeholders, particularly those directly affected and often branded as partners in the engagement project.

The reflection questions of Chilisa (2016, 313-328; Chilisa 2017, 824; Chilisa et al. 2017, 329; Chilisa and Mertens 2021, 241-253) and others (Mbah et al. 2021, 1-7) are relevant in the M&E framework for integrated community engagement. Adapting their line of questioning, I propose questions such as:

1) Does the research and engagement work have social relevance, and is it transformative?

2) Is the decolonisation and indigenisation intent explicit?

3) Does the research take a stance against the political, academic, and methodological imperialism of its time?

4) Does the research highlight potential areas of Western research incompatibility with local and indigenous epistemologies and convergence?

5) Is any concept or variable unique to the local phenomenon of study?

6) Does the unique concept or variable contribute to building a new theory or modifying existing ones?

7) Is there a local perspective, indigenous conceptual or theoretical framework that is used to inform a reflection on the specific context?

8) Are there unique ontological, epistemological, cultural and value assumptions that inform the study that differ from the globally generic or other cultural approaches?

9) What local or indigenous methods contrast to globally applicable generic strategies?

10) What are the locally relevant constructs that contrast to globally practical approaches that are generic?

11) Does the research contribute towards a new research approach that develops from an indigenous conceptual or theoretical perspective?

12) Does the research contribute towards documenting and restoring historical marginalised indigenous knowledge, cultures and values?

I believe that the onboarding, of any research project, particularly in rural areas, should include these questions. A decolonial monitoring frame revitalises and restores lost identities and value systems and legitimises the interacting knowledge ecologies that distinguish knowledge coproduction in the engagement space from the other (Chilisa et al. 2017, 326-339). It allows community members to ask questions, including the difficult ones, to which universities may not have answers. Therefore, the contest driver remains the desire for power over, gatekeeping and protectionism of the status quo.

The (im)possibility of this proposal is that an inclusive evaluation can only be deemed participatory if intervention recipients play an active role in developing the norms and standards. Furthermore, a critical attitude is necessary. Without a common, shared conceptual and operational understanding of what constitutes integrated community engagement in higher education, it is virtually impossible to engage in critical inclusive and emancipatory discussion of its central characteristics, methods, inputs, processes and expected outcomes and consequences.

 

CONCLUSION

In this article, I attempted to identify areas of (im)possibilities of M&E in a community engagement space by listing the problems and the contestations from a decolonial lens. I outlined the South African steady journey in community engagement to contextualise the M&E goals and problematise the persistent marginalisation of integrated community engagement discourse as a manifestation of academic imperialism, to which I put caution to its potential to spill into the proposed M&E framework. I further argued that the alignment of M&E goals should intentionally be reflective and reflexive to include societal impact measures and evidence that include the voices of the marginalised as well as restoration of epistemic justice. The aim of M&E of integrated community engagement should embrace transformation in the knowledge production project, entertain the multiple stakeholders, should bring together schedules, evidence as well quantitative and qualitative measures and testimonies of beneficiaries. Linked with the practices of M&E, is the desire for quality, to which I argue that such cannot be treated as benign. The marginalised should be brought into the discussion of quality, excellence, and societal impact, as a transformative and restorative practice and as an emancipatory co-learning process between the university and the community in context. Mutual respect, reciprocity, and equitable distribution of power and resources should be amongst the practice architect of M&E of community engagement in Higher Education. This should be intentional. In this regard, building capacities and proper resourcing cannot just be left to chance, as it has been for the most part of the two (2) decades in which community engagement steadily gained prominence in the country. I conclude, therefore, by also acknowledging the possibilities that integrated community engagement M&E may bring to advancing the scholarship of community engagement as a tool for transformative knowledge co-production practices.

 

REFERENCES

Adner, Ron. 2017. "Ecosystem as structure: An actionable construct for strategy." Journal of Management 43(1): 39-58.         [ Links ]

Adner, Ron and Rahul Kapoor. 2010. "Value creation in innovation ecosystems: How the structure of technological interdependence affects firm performance in new technology generations." Strategic Management Journal 31(3): 306-333.         [ Links ]

Bhagwan, Raisuyah. 2017. "Towards a conceptual understanding of community engagement in higher education in South Africa." Perspectives in Education 35(1): 171-185.         [ Links ]

Bamberger, Michael. 1991. "The politics of evaluation in developing countries." Evaluation and Program Planning 14(4): 325-339.         [ Links ]

Banks, Sarah and Hugh Butcher. 2013 "What is community practice?" Managing community practice: Principles, policies and programmes, 7-29.         [ Links ]

Bender, Cornelia Johanna Gertruida. 2008. "Exploring conceptual models for community engagement at higher education institutions in South Africa". Perspectives in Education 26(1): 81-95.         [ Links ]

Bhola, Harbans Singh. 2003. Social and cultural contexts of educational evaluation: A global perspective. In International handbook of educational evaluation, ed. Thomas Kellaghan and Daniel L. Stufflebeam, 397-416. Boston: Kluwer.         [ Links ]

Boyer, Ernest. 1996a. "The scholarship of Engagement." Journal of Public Service and Outreach (1): 11 -20.         [ Links ]

Boyer, Ernest. 1996b. "From scholarship reconsidered to scholarship assessed." Quest 48(2): 129-139.         [ Links ]

Braidotti, Rosi. 2019. "A theoretical framework for the critical posthumanities." Theory, Culture & Society 36(6): 31-61.         [ Links ]

Brink, Chris. 2018. The Soul of a University: Why excellence Is Not Enough. Policy Press.         [ Links ]

Chen, Huey Tsyh. 2006. "A theory-driven evaluation perspective on mixed methods research." Research in the Schools 13(1): 75-83.         [ Links ]

Chilisa, Bagele. 2017. "Decolonising transdisciplinary research approaches: An African perspective for enhancing knowledge integration in sustainability science." Sustainability Science 12(5): 813827.         [ Links ]

Chilisa, Bagele and Donna M. Mertens. 2021. "Indigenous made in Africa evaluation frameworks: Addressing epistemic violence and contributing to social transformation." American Journal of Evaluation 42(2): 241-253.         [ Links ]

Chilisa, Bagele, Thenjiwe Emily Major, and Kelne Khudu-Petersen. 2017. "Community engagement with a postcolonial, African-based relational paradigm." Qualitative Research 17(3): 326-339.         [ Links ]

Chilisa, Bagele, Thenjiwe Emily Major, Michael Gaotlhobogwe, and Hildah Mokgolodi. 2015 "Decolonizing and indigenizing evaluation practice in Africa: Toward African relational evaluation approaches." Canadian Journal of Program Evaluation 30(3): 313-328.         [ Links ]

Chouinard, Jill Anne and J. Bradley Cousins. 2015. "The Journey from Rhetoric to Reality: Participatory Evaluation in a Development Context." Educational Assessment, Evaluation and Accountability 27(1): 5-39. doi:10.1007/S11092-013-9184-8.         [ Links ]

Clayss - Centro Latinoamericano de Aprendizaje y Servicio Solidario, www.clayss.org.         [ Links ]

Connell, Raewyn. 2016. "Masculinities in global perspective: Hegemony, contestation, and changing structures of power." Theory and Society 45(4): 303-318.         [ Links ]

Cornwall, Andrea. 2008. "Unpacking 'Participation': Models, meanings and practices." Community Development Journal 43(3): 269-283.         [ Links ]

Cooper, David. 2011. "The University in Development: Case Studies of Use-oriented Research. " HSRC Press.         [ Links ]

Council on Higher Education. 2021. Framework for Institutional Audits. Pretoria: CHE.         [ Links ]

Council on Higher Education. 2004. Framework for Institutional Audits. Pretoria: CHE.         [ Links ]

Crawford, Paul and Paul Bryce. 2003. "Project monitoring and evaluation: a method for enhancing the efficiency and effectiveness of aid project implementation." International Journal of Project Management 21(5): 363-373.         [ Links ]

Cullen, Anne and Chris L. S. Coryn. 2011. "Forms and functions of participatory evaluation in international development: A review of the empirical and theoretical literature." Journal of MultiDisciplinary Evaluation 7(16): 32-47.         [ Links ]

Department of Education. 1997. Education White Paper No. 3: A programme on the transformation of higher education transformation. Pretoria: Government Printers.         [ Links ]

Department of Education. 1998. Higher Education Amendment Act. Pretoria: Government Printers.         [ Links ]

Department of Education. 2001. National Plan for Higher Education. Pretoria: Government Printers.         [ Links ]

DoE see Department of Education.         [ Links ]

Ebrahim, Alnoor and Kasturi Rangan. 2014. "What impact? A framework for measuring the scale and scope of social performance." California Management Review 56(3): 118-141.         [ Links ]

Etzioni, Amitai. 1995a. "The spirit of community, rights, responsibilities, and the communitarian agenda." New Statesman and Society 8(377): 34-35.         [ Links ]

Etzioni, Amitai. 1995b. The spirit of community: rights, responsibilities and the communitarian agenda. London: Fontana.         [ Links ]

Fals-Borda, Orlando. 1987. "The application of participatory action-research in Latin America." International sociology 2(4): 329-347.         [ Links ]

Galan-Muros, Victoria and Todd Davey. 2019. "The UBC ecosystem: Putting together a comprehensive framework for university-business cooperation." The Journal of Technology Transfer 44(4): 1311-1346.         [ Links ]

Gibbons, Michael. 2006. "Engagement as a core value in a mode 2 society." Paper presented at the CHE-HEQC/JET-CHESP Conference on Community Engagement in Higher Education. 3 to 5 September 2006, Cape Town, South Africa.         [ Links ]

Global University Network for Innovation (GUNI). https://www.guninetwork.org/. (Accessed 20 May 2022.         [ Links ])

Greenaway, Alison, Holden Hohaia, Erena Le Heron, Richard Le Heron, Andrea Grant, Gradon Diprose, Nicholas Kirk, and Will Allen. 2022. "Methodological sensitivities for co-producing knowledge through enduring trustful partnerships." Sustainability Science 17(2): 433-447.         [ Links ]

Gordon, Ray and David Grant. 2004. "Knowledge management or management of knowledge? Why people interested in knowledge management need to consider Foucault and the construct of power." Tamara: Journal for Critical Organization Inquiry 3(2): 27-38.         [ Links ]

Hagendijk, Rob. 2015. "Sense and sensibility: Science, society and politics as co-production." In Science and Democracy: Making Knwoledge and making power in the bioscience and beyond, ed. Stephen Hilgartner, Clark A. Miller and Rob Hagendijk, 238-256. New York: Routledge.         [ Links ]

Hart, Angie and Simon Northmore. 2011. "Auditing and evaluating university-community engagement: Lessons from a UK case study." Higher Education Quarterly 65(1): 34-58.         [ Links ]

HEQC see Higher Education Quality Committee.         [ Links ]

Higher Education Quality Committee. 2001. "The founding documents of the HEQC." www.che.ac.za/media_and_publications/heqc/higher-education-quality-committee-heqc-founding-document. (Accessed 29 June 2014).         [ Links ]

Hoppers, Catherine Alum Odora. (Ed.). 2002. Indigenous Knowledge and the Integration of Knowledge Systems: Towards a philosophy of articulation. Claremont: New Africa Books.         [ Links ]

Keane, Moyra, Constance Khupe, and Maren Seehawer. 2017. "Decolonising methodology: Who benefits from indigenous knowledge research?" Educational Research for Social Change 6(1): 12-24.         [ Links ]

Kemmis, Stephen. 2022. "Participatory action research: Methodologies for transformation." Webinar presentation, Collaborative universities (University of Mpumalanga, University of Venda, Central University of Technology). 29 June 2022.         [ Links ]

Kemmis, Stephen, Jane Wilkinson, Christine Edwards-Groves, Ian Hardy, Peter Grootenboer, and Laurette Bristol. (Eds.). 2014. "Praxis, practice and practice architectures." In Changing Practices, Changing Education, 25-41. Singapore: Springer.         [ Links ]

Khupe, Constance and Moyra Keane. 2017. "Towards an African education research methodology: Decolonising new knowledge." Educational Research for Social Change 6(1): 25-37.         [ Links ]

Limoges, Camille, Peter Scott, Simon Schwartzman, Helga Nowotny, and Michael Gibbons. 1994. The new production of knowledge: The dynamics of science and research in contemporary societies. London: Sage publications.         [ Links ]

Lusch, Robert F. and Satish Nambisan. 2015. "Service innovation."MIS quarterly 39(1): 155-176.         [ Links ]

Maas, Karen and Kellie Liket. 2011. "Social impact measurement: Classification of methods." In Environmental management accounting and supply chain management, ed. R. L. Burritt et al.,171-202. Dordrecht: Springer.         [ Links ]

Mbah, Marcellus, Ane T. Johnson, and Ferdinand M. Chipindi. 2021. "Institutionalizing the intangible through research and engagement: Indigenous knowledge and higher education for sustainable development in Zambia." International Journal of Educational Development 82(2021): 102355. https://doi.org/10.1016/jijedudev.2021.102355. 1-7.         [ Links ]

Markiewicz, Anne and Ian Patrick. 2015. Developing monitoring and evaluation frameworks. London: Sage Publications.         [ Links ]

Maughan, Catherine. 2012. "Monitoring and evaluating social impacts in Australia." Cooperative Research Centre for Remote Economic Participation (CRC-REP) Working Papers (Vol. CW003). Alice Springs: Ninti One Limited.         [ Links ]

Nkoana, E. and M. Dichaba. 2017. "Development and application of conceptual and analytic frameworks for community engagement at a South African higher education institution." South African Journal of Higher Education 31(6): 177-196.         [ Links ]

Nkoana, Elvis Modikela and Mpho Mildred Dichaba. 2016. "Are We Heading in The Right Direction? Assessing University-Community Engagement Dimensions of a South African Higher Education Institutions." Proceedings of the University of South Africa (UNISA) International Conference on Education, Towards Excellence in Educational Practices. Pretoria, South Africa. (Online).         [ Links ]

Patton, Michael Quinn. 1994. "Developmental evaluation." Evaluation Practice 15(3): 311-319.         [ Links ]

Schön, Donald A. 1995, "Knowing-in-action: The new scholarship requires a new epistemology. Change: The Magazine of Higher Learning 27(6): 27-34.         [ Links ]

Society for Participatory Research in Asia (PRIA). https://www.unescochair-cbrsr.org/. (Accessed 1 April 2022).         [ Links ]

Talloires network. https://talloiresnetwork.tufts.edu/who-we-are/.         [ Links ]

Tapia, Maria Nieves. 2012. "Academic excellence and community engagement: Reflections on the Latin American experience." In Higher Education and Civic Engagement, ed. L. McIlrath, A. Lyons, and R. Munck, 187-203. Palgrave Macmillan, New York.         [ Links ]

Vincent, Katharine, Meaghan Daly, Claire Scannell, and Bill Leathes. 2018. "What can climate services learn from theory and practice of co-production?" Climate Services (12): 48-58.         [ Links ]

Visman, Emma, Katharine Vincent, Anna Steynor, Irene Karani, and Emmah Mwangi. 2022. "Defining metrics for monitoring and evaluating the impact of co-production in climate services." Climate Services (26): 100297.         [ Links ]

Vogel, Coleen, Anna Steynor, and Albert Manyuchi. 2019. "Climate services in Africa: Re-imagining an inclusive, robust and sustainable service." Climate Services 15: Article 100107. 10.1016/j.cliser.2019.100107.         [ Links ]

West, Simon and Caroline Schill. 2022. "Negotiating the ethical-political dimensions of research methods: A key competency in mixed methods, inter-and transdisciplinary, and co-production research." Humanities and Social Sciences Communications 9(1): 1-13.         [ Links ]