SPECIAL EDITION

 

Quality Research Management Improves Design Research Effectiveness

 

 

J.E.W. Holm^*; G.P.R. van der Merwe

Faculty of Engineering, Northwest University, South Africa

 

 

---

ABSTRACT

Design research projects are often more complex than initially perceived due to a requirement for novelty, the creativity inherent in the design process, and the limited experience of novice researchers. Also, expectations from institutions and external funding stakeholders must be met. Therefore, a framework was needed to support the research effort in the form of quality research management. The focus of this framework is primarily to align stakeholders. To this end, engineering management methods were consulted to define a quality research management framework that provides review baselines, requirements traceability, and a general process model. The framework is based on the well-known 'house of quality' principle that provides requirements traceability. A series of matrices was developed, with a research validation matrix resulting from the combination of traceability matrices. This matrix allows the allocation of research challenges to a need, concept solutions to research challenges, and specific research solutions to concept solutions. Also, information sources are allocated to challenges and solutions to verify their authenticity. It is thus possible to visualise clearly a summary of the research project. Additional benefits include improved communication, visualisastion of progress, traceability of requirements, and verification and validation, which illustrates the value of quality research management.

---

OPSOMMING

Ontwerpsnavorsingsprojekte is dikwels meer kompleks as wat aanvanklik begryp word weens die vereiste vir oorspronklikheid, die kreatiwiteit inherent tot die ontwerpsproses en die beperkte ervaring van nuwe navorsers. Verwagtinge van instansies en eksterne befondsers moet ook aangespreek word. Daarom word 'n raamwerk benodig, in die vorm van kwaliteit navorsingsbestuur, om die navorsingspoging te steun. Die fokus van die raamwerk is hoofsaaklik om belanghebbendes te belyn. Met hierdie ten doel was ingenieursbestuursmetodes aangewend om n kwaliteit navorsingsbestuur raamwerk te definieer wat hersieningsbasislyne daarstel, vereistes naspeurbaar maak, en n algemene prosesmodel beskikbaar stel. Die raamwerk is gebaseer op die bekende 'house of quality' beginsel wat naspeurbaarheid van vereistes daarstel. 'n Reeks matrikse is ontwikkel met n navorsingsvalidasiematriks gedefinieer as die samestelling van naspeurmatrikse. Hierdie matriks ondersteun die toedeling van navorsingsuitdagings aan n behoefte, konsep-oplossings aan navorsingsuitdagings en spesifieke navorsingsoplossings aan konsep-oplossings. Informasiebronne word ook toegedeel aan uitdagings en konsep-oplossings om die oorspronklikheid daarvan te verifieer. Dit is dus moontlik om die projek opsommend te visualiseer. Addisionele voordele sluit verbeterde kommunikasie, visualisering van vordering, naspeurbaarheid van vereistes, verifikasie en validasie in, wat die waarde van kwaliteit navorsingsbestuur illustreer.

---

 

 

1 OVERVIEW

Quality is known to imply that a process produces outputs aligned with requirements and expectations [1], [2], [3]. Thus, following a quality approach in research management should, ideally, result in defined research process tasks and sequences, with measurement and control to ensure that research outputs are produced within specification and to expectation.

A surprisingly limited offering of methods exists for the quality management of grass roots research effort (that is, of individual researchers and their study leaders) [4], [5], although general project management principles have been applied to research management to good effect [6], [7]. From the existing literature [8], [7], [9], [10], [11], [12], the levels at which research management is done include the following:

• Institutional level: The level at which a research strategy is typically aligned with institutional strategy and implemented using programmes in an organisational structure;

• Operational level: The level at which research teams and individuals plan and manage research in defined research projects to align with institutional programmes;

• Specialist level: The level at which specific research tasks are defined and specialised research methods are applied to provide outputs aligned with project goals and objectives.

The methods discussed in this paper apply to work done at operational level, where limited research specifically on quality research management has been published [13], [14]. According to Kajaste [13], quality in research (in general) had not received much attention in Finland, with a need for quality research management specifically for research, development and innovation (RDI)-type research projects. The link between specialist level research and institutional level research is provided by research management at operational level, for which there is a need.

This paper thus addresses the need for quality research management at operational level, specifically applied to design research. To this effect, useful research frameworks exist to provide both an environment and a methodology to guide design research, such as design science research (DSR) [15] [16] and action design research (ADR), of which the derived elaborated ADR (eADR) is a useful process model [17].

From personal experience in research leadership, it was observed that the researcher is often challenged to focus her / his research where a misalignment between research project leader and researcher(s) sometimes occurs, resulting in time delays or change of research project scope. Documenting research work becomes a challenge, researcher motivation may decrease, and communication with external evaluators and examiners sometimes lacks clear context and focus. These are familiar challenges found in any project, let alone innovative applied research projects that are inherently rich in unknowns and project risk [1]. These familiar challenges may be addressed using quality management principles in order to increase research effectiveness.

This paper provides a quality framework, derived from engineering and quality management principles, for the management of applied research at an 'operational' or grass roots level where researchers interact with study leaders, literature, peers, and examiners inside a research environment. By following an action research approach, a practical method for tracking research requirements, ensuring validation, assisting with process and project management, and improving communication is provided to assist both researchers and study leaders in this complex endeavor. A management tool - a research validation matrix - is presented and discussed to close out this article.

 

2 BACKGROUND

From a systems perspective, it is instructive to define a general operational research environment in terms of its elements and respective interactions between elements to provide context, as shown in the figure below:

It was found from different research studies that subscribing to the DSR paradigm principle adds significant value to design research [15], [18], [19]. Although DSR was primarily focused on information systems (IS), its utility has been demonstrated in research projects that produced artefacts quite different from purely IS artefacts where the principles of the DSR process were applied (refer to [20], [21]). Valuable research outputs of most research processes include: thesis documents and publications; artefacts in the form of demonstratable entities; and meta-artefacts that all add to a shared knowledge base. This 'body of knowledge' includes knowledge generated from research, as well as prior knowledge used to benefit both current and future participants.

With reference to Figure 1, the goal of the researcher is to deliver a thesis document and publications (as well as artefacts) as a result of the design research process (indicated in the diagram as interface 2) in alignment with the research focus of the study leader and institution within which research is conducted (interface 1). The thesis document and artefacts are most often evaluated by independent examiners (interface 3) and peers (interface 5) to control quality. In this research environment, the student also interfaces with the literature and other related enablers, as shown in the diagram (interface 4). The diagram also includes an interface between the study leader and the institution (interface 6) for the alignment of grassroots research efforts with institutional strategy, and an interface between the researcher and her / his benefactors, when present (interface 7), as these stakeholders also benefit from quality research. Commercial interfaces are not shown, as they fall outside the scope of this discussion. Thus several participants are present in the operational research environment, each with its own expectations and requirements.

The environment above forms the core system for producing grassroots research outputs and maintaining research momentum. The seemingly simple system becomes more complex when research processes and methods, management of the process, and most importantly, communication across interfaces are considered.

In design research, a process is followed in which research challenges are systematically identified and addressed to provide verified and validated solutions to these challenges. This process involves elements of the research environment architecture (from Figure 1 above) throughout the life cycle of a research project. Detailed treatment of a design research process model was done in prior research [19]. This process model may also be described by an abstracted research project process model, as outlined below, with the understanding that this is a general outline that may differ between projects. Apart from using generalised task descriptions in the process model below, a minor adjustment was made to the original model [9] - namely, specifically to distinguish between analysis and synthesis, as shown in tasks B and C below:

A. Inception / definition phase

a. Definition and validation of a clear research goal aligned with institutional objectives, benefactor requirements, and study leader's focus area(s) and project(s) (across interfaces 2, 6 and 7);

b. Derivation and validation of associated research objectives (challenges) from the main goal; typically more than one research objective or challenge will be derived (interface 2);

c. Identification of a research process model and definition of the applied research process model (such as eADR) to ensure a clear understanding of the way forward (pure systems engineering (SE) development, eADR, or similar);

B. Analysis phase

a. Execution of a literature study directed to address research challenges, with each literature topic identified to cover each and all research challenges (interfaces 2 and 4);

b. Further analyses of research challenges with the purpose of breaking the problem down into smaller subsets of challenges commensurate with the main research challenges;

c. Documentation of literature analyses and inferences in order to perform synthesis from the literature study (interface 1);

C. Solution / synthesis phase

a. Definition of research solution(s) to address each main research challenge specifically, often with more than one solution to a challenge (interface 2);

b. Application of a design research methodology to address research challenges and their associated research solutions in a systematic way (interface 4);

c. Design and construction of an artifact or meta-artefact by means of a formal design methodology based on design theory;

D. Test and evaluation phase

a. Generation of sets of verification results from tests, experiments, surveys, and other verification methods, including processing of results and data (interface 1);

b. Evaluation and verification of results to accepted standards, with inferences and conclusions drawn from analyses (interface 4);

E. Communication phase

a. Documentation of results in theses, generally known academic and technical publications, online, and other publishing platforms (interfaces 1 and 5);

b. Communication of research and results to peers and examiners for critical review and quality control (interface 3);

c. Defence of reviewed theses and other publications, often in public and in peer-reviewed publications (interfaces 3 and 5);

F. Close out

a. Final adjustment and associated change management (also done throughout the research process). This is a critical task, as final objectives cannot always be clearly defined at the onset of a new or innovative research project.

The research process above must be managed, which is a complex task for a number of reasons, including, among others (from personal experience): influx of new students from diverse backgrounds who must be educated in research methodology and the environment; goals, objectives, results, and solutions that change as new knowledge becomes available; lack of funding, which introduces constraints that, in turn, demand efficacy; and research time that is often limited. Experienced research leaders have developed research process management methodologies that serve them well; but new study leaders and researchers are sometimes perplexed by the complexities of research [7]. Complexity of this nature requires a quality research management framework that allows for effectiveness, efficiency, and continuity, as found in systems where quality is evident.

A quality research management (QRM) framework is the topic of this paper. It presents a number of desired characteristics associated with a research environment, as defined in Figure 1 and the generic research process above. With the focus purely on the technicalities of design research (i.e., financial aspects are excluded for now), desirable characteristics of a quality research management framework are provided below:

• A clearly defined research process and associated research management methodology must be available and visible to all role players, for which the DSR paradigm was found to be highly useful and effective in directing research, although other methods may be used;

• Unambiguous definition and alignment of goals and objectives must be achieved by the student and study leader - i.e., clearly agreed definition of research project scope;

• Clear communication of expectations from both study leader and student perspectives must be performed to ensure alignment;

• Progress must be managed by means of a work plan with defined milestones and deliverables, visibly governed by progress monitoring and tracking;

• Research goals and objectives must be systematically converted into solutions within constraints and by using enablers (as shown in Figure 1), allowing for traceability as a matter of importance;

• Change must be managed (as new knowledge becomes available from the research) to ensure convergence towards scientifically sound results and solutions;

• Validation and verification of research play important roles and require planning, execution, and monitoring to ensure quality work and deliverables of acceptable standard;

• Communication of research challenges, methodology / process, results, and solutions to provide examiners and peers with clear-cut information for review;

• Continuity must be ensured so that the momentum of research projects is maintained by means of comprehensive documentation inside a familiar context.

The above list of desired characteristics is not unique to research alone, as these characteristics are present in most development projects (i.e., projects that require systems engineering [2] and project management [1]).

 

3 A QUALITY RESEARCH MANAGEMENT FRAMEWORK

In order to provide a quality research management framework, different focus areas were consulted that, when combined, address each of the desired characteristics above. As the research management in this paper is defined at operational level, and the focus is on design research that delivers artefacts, it was deemed appropriate to use proven engineering management [1] [2] [6], quality management [2] [14] [22], and research methods [11] [15] [17] [18] for a QRM framework. Thus relevant disciplines were identified, and include: quality management; systems engineering; project management; and (importantly) communication. The methods and techniques applicable to operational level management from these disciplines are discussed and contextualised to define the QRM framework below. The combination of these methods and techniques resulted in a research validation matrix (RVM) that captures the essence of quality research management framework as proposed after this discussion.

3.1 Factors influencing the QRM framework

The factors above are discussed in the sections that follow. Project management (as such) is well-known and has been extensively documented elsewhere [1,4]. As a result, the following areas will be discussed in more detail: (i) quality management, (ii) change management, (iii) requirements management, and (iv) communication. The value of requirements traceability and visibility will be highlighted at the end of this discussion.

 

 

3.2 Quality management

' Quality management' refers to a process whereby client (or customer) needs are converted into solutions in a systematic, traceable manner to deliver an end-product that meets specifications and expectations. Quality, with specific reference to the 'voice of the customer' and a house of quality (HoQ) methodology, provides excellent tools for the management of expectation and traceability of requirements ('what' must be done) allocated to solution characteristics ('how' it will be done) [1], [2], [3]. The customer's voice in this case will be that of the research study leader, whose voice will in turn be aligned with institutional research goals and objectives, the real-world environment, and her / his own research interests.

A modified HoQ is shown in Figure 2, where requirements are shown in rows and solutions are shown in columns. The relationships between requirements and solutions are shown as entries in the matrix, and correlations between possible solutions are shown in the triangle at the top. Clearly, one should endeavor to minimise correlation between solutions by selecting solutions that are independent and / or largely uncorrelated. Additional information not shown on the matrix includes weights associated with requirements and solutions, existing or competing solutions and their relevance to requirements, and the relative effectiveness of proposed solutions; these will not be used in this discussion for simplicity's sake. The HoQ matrix presented here has thus been contextualised specifically for research.

 

 

A house of quality thus provides a technique to ensure that research requirements are addressed by research solutions using a relationship matrix [2]. The relationships between requirements and solutions can thus be visualised and traced. Adding traceability allows a study leader an opportunity to ensure that, for example, a correct set of research requirements is being addressed (validated research) in a comprehensive and traceable manner (verified research).

3.3 Change management

Change is always present in projects of a developmental nature, such as research projects in which new concepts and solutions are developed. Change is unavoidable (and necessary) in research projects, as it is often a result of creativity; but just as often, change results in unplanned delays and loss of project focus. For the sake of research project control, it is necessary to define specific milestones in the form of time-based review points where deliverables are verified, progress is recorded, and change is controlled in a structured manner. By controlling change, variations in scope, schedule, and cost (where applicable) can be minimised while not being overly restrictive of creativity.

Baselines are used to record the status of a project that follows a defined research process. For the generic process defined above, the following baselines are defined:

• Definition baseline - The project inception phase results in a set of research challenges that clearly defines the problem as a result of an analysis. A clear, valid problem must be presented as well as a plan for the following phase. Change at this stage is fairly common, and does not present a major risk;

• Analysis baseline - Literature studies confirm the validity of the research problem and add to the solution by identifying useful existing knowledge and methods. From the literature study, a document is presented that confirms the validity of the research problem and supports the originality of a concept solution. A plan is also presented for the next phase, considering that the concept solution may change in an iterative manner;

• Synthesis baseline - After an integrated solution has been defined in the form of a set of individual solutions to the research challenges, results are generated and presented. The proposed solution, results generated from the solution, and analyses thereof are documented and proposed with a plan for the finalisation of the research project. Change at this stage presents a major risk;

• Validation baseline - Communication of research efforts and results is performed such that the validation and verification of the research methods, studies, and solutions are presented. A final thesis document or the like is presented for examination and defense purposes. Final changes after peer review and examination are included, but changes to scope and schedule should not be allowed unless absolutely necessary.

Baselines are sets of documents that accurately reflect the status of the research process, including studies, concepts, plans, and artefacts. Once the status has been recorded at a milestone, change to that baseline is allowed under controlled conditions; this maintains alignment between researchers and managers and minimises diversions and loss of focus. Baselines are set at milestones that correspond with the research process, and are placed under change control.

Importantly, baselines do not necessarily follow from a linear research process, as research is an iterative process in which research challenges and solutions are redefined as new information comes to light. As a result, change control should not restrict the creative process, but must aim to provide for structured change. Baselines are thus used to ensure milestones are reached; otherwise the research becomes divergent and unfocused.

3.4 Requirements management

Successful projects have a form of requirements management, from the point where expectations are formed by both study leader and researcher (and possibly other stakeholders) at the onset of a research project, through to a systematic, traceable conversion of requirements to final solutions.

3.4.1 Inception / definition phase and baseline

When used correctly, requirements may be traced using a matrix similar to the HoQ matrix used for general quality management. At the onset, a single need is addressed in the form of a research goal - thus, one need for one research project. More detailed requirements (derived from the research goal) are captured in the form of research challenges (on the columns of the matrix in Figure 3 below) that will be addressed with research solutions that are aligned with the challenges. Importantly, research challenges are often incorrectly regarded as objectives - this may be confusing, as research problems and solutions should be separated. Therefore, challenges should be stated as 'shortfalls' or opportunities pertaining to a real-world problem to ensure a clear separation between problem and solution.

 

 

By stating clear challenges, a definitive baseline of research goal and associated challenges is created early in the research project, and a clear distinction is made between challenges and solutions. Research must be done initially to derive shortfalls / challenges from the research problem; this is typically supported by the research leader's own subject expertise, academic literature, case studies, and observations, among others. The result of this initial study is a matrix, as shown in Figure 3.

A clearly defined research problem, in the form of research challenges, is critical at the onset of a research project. Challenges should also be independent, as can be verified from the correlation between challenges shown in Figure 3 above. Validated research challenges will form the input to the following steps in the research process, and must thus be comprehensive and unambiguous. It is also necessary to show that research challenges have not been fully addressed by existing research, which may be shown by linking published research from a literature study to the proposed research challenges, using a relationship matrix.

A review at this stage allows the researcher to prepare an informed research proposal, with focus on the following aspects:

• Sources of origin must be checked for validity, where validity requires recorded and preferably peer-reviewed information;

• Challenges must be verified to be as independent as possible to prevent duplication, overlap, and possible confusion;

• The nature of a challenge is that of a 'problem' to be addressed, as opposed to an objective or solution - a challenge must thus be expressed correctly to make up a clear definition of the overall research problem;

• Challenges must be addressed in a comprehensive way. That is, the scope of the research should be limited to suit the level of research, but must still address the total research problem.

3.4.2 Analysis phase and baseline

Besides a systematic breakdown of research challenges, an analysis requires a literature study that will ensure that the following focus points are addressed:

• A valid research problem has been defined (the literature actually starts at the onset of the research project and adds value throughout the project);

• The research problem (in the form of challenges) is further analysed and understood in context;

• The researcher and study leader are informed on relevant literature topics - that is, all relevant topics have been covered;

• Potential solutions are identified to the defined research challenges as part of a final integrated solution; and

• The solution is not trivial and has not been presented elsewhere (i.e., the solution is mostly unique).

Analysis of the problem not only leads to insight into the research challenges, but also inevitably leads to solutions during the literature study - a situation that can blur the lines between problem and solution. Therefore, it is instructive to separate research challenges from concept solutions during the literature study / analysis phase; this is done in an iterative manner.

The matrix below (Figure 4) shows how a literature study informs both the analysis and the synthesis sides of the research process. Research topics are identified and arranged in the rows of the matrix. Arrows that point upward and downward show either that a literature topic (located in the rows of the matrix) verifies a research challenge at the top end of the matrix, or that a literature topic adds value to a concept solution at the bottom end. The direction of the arrow is thus used to show whether a topic confirms the challenge or informs a solution. Note that each research challenge at the top of a column is met by a matching concept solution at the bottom of that column.

 

 

Literature topics are not limited to academic publications, and may include other valid literature on a topic. (If a literature topic has not been documented, it effectively does not exist, and should be avoided.) The researcher should be using prior research publications in this phase: all new research (original) outputs and efforts will be part of the synthesis phase.

The baseline at the end of the analysis phase comprises a literature study, including references and an analysis of the literature as defined by the research topics, and a matrix that shows how literature topics have contributed towards challenges and concept solutions. In doing so, a clear overview of the complete analysis of the problem is provided (combined with concept solutions that could also be defined in this phase).

Concept solutions are high-level solutions to the stated research challenges, and are stated in general terms to ensure that focus is maintained at a high level. This activity is often referred to as 'synthesis from literature'. The concept solutions are further broken down into smaller, individual solutions in the synthesis phase that follows below. As a result, the researcher is advised to refrain from detailing solutions at this stage.

A review should be held to ensure the researcher has achieved the following:

• At this stage, relevant research topics must be identified. Each topic should address at least one research challenge, concept solution, or background to the study. Literature topics must thus be linked to the study to add real value, as opposed to providing an unrelated maze of information;

• It must be shown that all literature topics have been covered in a comprehensive manner. Ideally, a full view of the literature study topics should be provided - although it often happens that topics are uncovered as the research progresses. Nonetheless, at the end of the literature study, topics that describe the research challenges should be as encompassing as possible to limit the risk of an incomplete study;

• Source validity is critical, as all sources should be of high integrity. It is not uncommon to find unvalidated literature on the internet, and other dubious sources that will cast doubt on the validity of solutions that emanate from the literature study. Therefore, the validity of sources and topics must be ensured as a matter of importance;

• Literature topics must be defined at a high level in the matrix. Each literature topic will undoubtedly contain numerous references to individual articles, books, and other sources of information. Detail should be limited in this matrix, as detail on each research topic will be dealt with comprehensively under the literature study section most often contained in the thesis document;

• Literature topics must be linked to either challenges or solutions - this allows for research focus.

3.4.3 Synthesis phase and baseline

The synthesis of specific solutions from concept solutions may also be managed using a HoQ matrix to link an individual concept solution to one or more specific specific research solutions. This matrix is shown in Figure 5 below. 'Synthesis' is a general term for the generation of solutions using literature study results and creativity as inputs to the solution / design effort in an iterative manner. This phase is usually not linear due to new findings influencing prior knowledge, assumptions, theories, and the like. This is well-known, but does not negate the value of setting baselines and aiming for specific objectives.

 

 

In DSR, specifically, both theory and artefacts are typically produced as specific solutions. Specifically, artefacts are produced in the form of constructs, methods, models, or instantiations; these may be used as specific solutions [23].

In its presentation in the matrix, specific solutions should follow a logical sequence, as this simplifies reading and understanding. For example, in the case of a development project, problem analysis (iflisted as a research challenge) should precede design and implementation (called 'instantiation' in the DSR paradigm). Alternatively, in the event of a survey, definition of the survey objectives should precede design of the questionnaire and analysis of results. In studies where articles are published in order to complete a doctoral study, articles and their evolutionary development into solutions may be listed and described, which allows the reader an opportunity to follow a storyline that is linked to research challenges through concept solutions.

Specific solutions may include, for example, any of the following (and potentially more):

• Effort in the form of processes (for example, surveys or experiments) that were followed and used to achieve specific outcomes. This may also show originality, as the selection of methods and techniques may be singular. Experiments and analyses methods are sometimes mentioned as work effort, but may in fact be part of a unique contribution in the form of meta-artefacts;

• Physical artefacts resulting from research, specifically when using DSR as a research methodology (including software code, hardware, and sometimes a complete system). An artefact may be the complete integrated solution, or a sub-set of the overall integrated solution;

• Methods and techniques that had been derived from the research. These will contribute to the body of knowledge that will be used in future research. Often, methods, tools, and techniques are the main results from research, to be applied in practice as instantiations, which may form part of future design theories;

• Results, analyses of results, and all effort that contributed towards the creation of an overall solution. Deductions, evaluations, conclusions, and other creative output often follow from experiments;

• Publications, often in journals or at conferences, as well as reports and other documented outputs, may result from creative effort applied to specific solutions. These may be listed individually or in a combined manner to show, for example, a complete case study as one specific solution;

• Integration of smaller solution elements that, when combined, form the overall solution made visible by the matrix. This valuable contribution is often omitted from thesis documents when they could have highlighted a unique contribution in the form of integrated effort and a solution made up of numerous integral smaller parts.

3.4.4 Validation phase and baseline

There is often disagreement about what is meant by 'research validation'. However, considering the matrices presented in the definition, analysis and synthesis phases, it is evident that a definition of 'research validation' lies in the combination of these matrices. A combined matrix, defined here as a research validation matrix (RVM), is shown in Figure 6. It is thus put forward that validation should be considered in an integrated manner, as follows:

 

 

• The initial research need must be validated to ensure that a non-trivial, relevant problem is addressed in the research, which is a requirement for DSR, for example. This is mostly the responsibility of the study leader, but at advanced levels of research (PhD studies), the researcher will share this important responsibility;

• Different challenges, as derived in the definition phase, combine to underly the research need in a systematic manner. Each research challenge is confirmed, and thus verified, by means of one or more sources that validate the challenge. It may be the case that a need has been addressed (fully) or that a need does not exist - this must be verified by confirming (verifying) each and all challenges;

• In the analysis phase, literature topics were identified and listed. The validity of the literature must be verified, after which each verified literature topic will confirm the existence of a valid challenge, or may contribute to the creation of a solution. Should one literature topic confirm all concept solutions, the possibility of a unique contribution reduces, and the research is at risk. However, literature topics should always contribute to concept solutions in a verified manner;

• In the synthesis phase, which had started during the analysis phase in a concurrent manner, creative input is usually required to distinguish valuable research contributions from esoteric or obvious contributions. Even when experiments are conducted, the aim should be to explore and find new information;

• In the validation phase, the authenticity of all information sources, work effort, and original contributions is validated by means of (i) specifically indicating original contributions under 'specific solutions', (ii) verification of all information sources, including prior knowledge from the body of knowledge, (iii) validity of research methods and techniques, and (iv) relevance and rigour of artefacts in the DSR paradigm (when used).

The RVM is shown in Figure 6, with descriptions of each of the sections in the matrix provided below for ease of understanding:

• At the top, the matrix from the definition phase is shown (flipped) to show all research challenges (equivalent to research requirements). Arrows in the matrix's columns show which information sources verify research challenges;

• The matrix from the analysis phase is positioned below the research challenges to show the relevance of the literature sources (defined as literature topics) when describing research challenges and concept solutions. Arrows pointing upwards are used to show relevance to research challenges, and arrows pointing downwards show relevance to concept research solutions;

• At the bottom of the matrix are specific solutions to the concept solutions, as defined from the synthesis phase. Arrows pointing upwards show how specific solutions contribute towards concept solutions.

It is evident that specific solutions can now be traced to concept solutions, which in turn are linked to research challenges, which are linked to a single research challenge or need. The need, challenges, and solutions are also linked to the literature, work effort, and creative input in the matrix.

3.5 Communication

The communication of research results in thesis documents is often a challenge to novice researchers. However, by using traceability and visualisation as shown above, the research (structure, effort, contributions) is represented in a single matrix. This simplifies the task of reviewers and, generally, the reading of complex thesis documents. A storyline follows from top to bottom in a systematic manner, with traceability of solutions (in a 'what-how' manner), verification of information sources, and visibility of original contribution evident in the matrix. Communication between the study leader and researcher is enhanced as goals are clear, methods visible, baselines defined, and progress can be made visible as the matrix is populated from the top down (the authors used red, amber, and green in the matrix entries during projects to show and track progress, to good effect).

 

4 CONCLUSION AND FUTURE RESEARCH

The complexity of research projects is often underestimated, resulting in a misalignment between stakeholders and research project delays. From the wealth of engineering management methods, techniques and methods were used to provide a framework for quality research management (QRM). The framework relies upon configuration management principles for the definition of baselines, and requirements management to provide traceability and visibility for quality purposes. House of Quality matrices are adapted to show how a single research need is converted into research challenges in the definition phase, followed by the allocation of the literature to both research challenges and concept research solutions in the analysis phase, leading to specific solutions for concept solutions in the synthesis phase, with validation relying upon inputs from all prior phases in the validation phase.

The value of the QRM framework lies in the visualisation of results from each phase, combined with the traceability and verification of information sources and creative input. This allows researchers and their study leaders to track progress, verify authenticity, communicate effectively and, most importantly, align expectations with solutions. This framework has been successfully applied in Master's and Doctoral studies (refer to [24] [25] [26] for Master's and [21] [27] for Doctoral studies) and allowed research projects to be focused and on time, and to address quality requirements. The quality research management framework presented in this paper was aimed specifically at design research, and generalisation is required for the application of this framework across disciplines. To this end, future research will include application of the quality management principles presented here to research projects other than design research. As it stands, the management methodology presented here addresses the need for quality design research management at operational level, and thus provides a framework for both design research quality assurance and control.

 

REFERENCES

[1] Nicholas, J. and Steyn, H. 2017. Project management for engineering, business and technology. New York: Routledge.         [ Links ]

[2] Blanchard, B.S. and Fabrycky, W.J. 2014. Systems engineering and analysis, 5th ed. Pearson, Edinburgh Gare, Harlow, Essex.         [ Links ]

[3] INCOSE. 2015. Systems engineering handbook, 4th ed. Wiley, Hoboken, NJ.         [ Links ]

[4] Gaaloul, K. and Molnar, W. 2014. Research methodologies in enterprise engineering: Insights from a workshop, in IEEE International Workshop on Advanced Information Systems for Enterprises, Tunis, Tunisia, 2014, November 10-12.         [ Links ]

[5] Tedrem M. and Verhagen, H. 2014. Research methodology education in computing: Arrangements and results from two courses, in IEEE Frontiers in Education Conference (FIE) Proceedings, Madrid, Spain, 2014.         [ Links ]

[6] Mustaro, P.N. and Rossi, R. 2013. Project management principles applied in academic research projects. Issues in Informing Science and Information Technology, 10, 325-340.         [ Links ]

[7] Howard Hughes Medical Institute. 2006. Making the right moves - A practical guide to scientific management for postdocs and new faculty. Chevy Chase, Maryland: Howard Hughes Medical Institute and Burroughs Wellcome Fund.         [ Links ]

[8] Sahu, P.K. 2013. Research methodology: A guide for researchers in agricultural science, social science and other related fields. Nadia, India: Springer.         [ Links ]

[9] Vachan, B. 2012. Applying project management principles to research projects in a health setting. The Radiographer, 59(1), 5-7.         [ Links ]

[10] Liu, Y., Yao, Y., Zhang, X., Liu, B. and Ding, C. 2015. Design of research management system based on workflow and rapid development platform technology, in International Conference on Estimation, Detection and Information Fusion, Habin, China, 2015.         [ Links ]

[11] Johnson, A. 2013. Improving your research management. Amsterdam, The Netherlands: Elsevier.         [ Links ]

[12] Ruiz, J.L., Arellano, C.C. and Segundo, L.M. Management of scientific and technological research, in International Conference on Computing Systems and Telematics (ICCSAT), Xalapa, Mexico, 2015.         [ Links ]

[13] Kajaste, M. 2018. Quality management of research, development and innovation activities in Finnish universities of applied sciences. Quality in Higher Education, 24(3), 271-288.         [ Links ]

[14] Flick, U. 2007. Managing quality in qualitative research. New-Delhi, India: Sage Publications.         [ Links ]

[15] Hevner, A., March, S., Park, J. and Ram, S. 2004. Design science in information systems research. MIS Quarterly, 28(1), 75-105.         [ Links ]

[16] Hevner, A. 2007. A tree cycle view of design science research. Scandinavian Journal of Information Systems, 19(2), 1-6.         [ Links ]

[17] Mullarkey, M. and Hevner, A. 2019. An elaborated action design research process model. European Journal of Information Systems, 28(1), 6-20.         [ Links ]

[18] Gregor, S. and Hevner, A. 2013. Positioning and presenting design science research for maximum impact. MIS Quarterly, 37(2), 337-355.         [ Links ]

[19] Peffers, K., Tuunanen, T., Rothenberger, M. and Chatterjee, S. 2007. A design science research methodology for information systems research. Journal of Management Information Systems, 24(3), 4578.         [ Links ]

[20] Van der Merwe, G., Holm, J. and Hoffman, A. 2017. Activity-based risk acquisition of electronic safety equipment for mines. Journal of the Southern African Institute of Mining and Metallurgy (SAIMM), 117(6), 517.         [ Links ]

[21] Viljoen, D. and Holm, J. 2017. A life cycle model for the development of airborne electronic equipment. South African Journal of Industrial Engineering, 28(3) (Special Edition), pp 89-104.         [ Links ]

[22] Anderson, C., Rungtusanatham, M. and Schroeder, R.G. 1994. A theory of quality management underlying the deming management method. Academy of Management Review, 19(3), 472-509.         [ Links ]

[23] Baskerville, R., Baiyere, A., Gregor, S. and Hevner, A. 2018. Design science research contributions: Finding a balance between artifact and theory. Journal of the Association for Information Systems, 19(5), 358-376.         [ Links ]

[24] Du Preez, R. 2017. Deployment, re-engineering and risk analysis of a C-band weather radar to build local capacity in South Africa. Potchefstroom: North west University.         [ Links ]

[25] Nel, R. 2015. Synthesis and evaluation of an autonomous neutron monitor system for use in a very low temperature environment. Potchefstroom: North west University.         [ Links ]

[26] Weakley, B. 2018. Research and development of a real time measurement and evaluation system for SAG Mill Liner Wear. Potchefstroom: North west University, 2018.         [ Links ]

[27] Van der Merwe, G. 2014. A risk-based approach to the acquisition of electronic safety equipment for mines. Potchefstroom: North west University.         [ Links ]

 

 

* Corresponding author: johann.holm@nwu.ac.za