Four challenges in selecting and implementing methods to monitor and evaluate participatory processes: example from the Rwenzori region, Uganda

Get Complete Project Material File(s) Now! »

Undertaking the research journey: a typology of approaches for complex systems research

The complexity of social-ecological systems requires researchers to adopt a complex systems research approach. This paper focuses on complex systems research which is transdisciplinary, engaged, in that researchers are embedded in the observed system, applied and using mixed methods. Various research approaches exist which may guide complex systems researchers in their investigation. The breadth of options available can be illustrated by a “research journey”. However, few guidelines exist to help researchers undertake such journey. Building on existing theoretical debates about approaches, methods, paradigms and assumptions, this paper aims at helping complex systems researchers to undertake their research journey with more confidence. Specifically, this paper provides a typology of research approaches which can guide complex systems researchers, especially those beginning their research career, to develop reflexivity and clarify the key assumptions underlying their research, check the coherence of these assumptions, and situate their research in the array of existing research approaches. By identifying which approaches bear similar or contradictory philosophical assumptions to their own, complex systems researchers can then undertake more informed reading and use of the literature. They can also more easily communicate their findings and cooperate with researchers with similar interests and different skill sets. Seven research approaches are included in the typology: case study, grounded theory, participatory research, feminism, action research, intervention research and evaluation research. An illustration of the potential use of this typology is provided through the example of the first author’s doctoral research. The paper concludes on the added-value and limits of the typology and emphasizes aspects of coherence, reflexivity and transparency in complex systems research.

Introduction

Social-ecological systems are complex (Berkes, Colding, & Folke, 2003). They are characterized, among other features, by cross-scale interactions, nonlinear feedback, uncertainty, resilience to change, self-organization, and emergence (Gunderson & Holling, 2002). As a result, researchers face enormous challenges when confronting this complexity in their work (Walker et al., 2002). Traditional scientific methods, founded on a reductionist paradigm, are a poor fit for complex systems and situations (Berkes et al., 2003). As a result, researchers willing to understand the relationship between people and the environment need to adopt a complex systems research approach. Complex systems research is typically characterized by transdisciplinarity (Cundill, Fabricius, & Marti, 2005), self-reflection (Preiser, 2012) and embeddedness of the researcher in the observed system (Midgley, 2000). Complex systems researchers also increasingly often seek to develop applied research (Rogers et al., 2013), where they can generate both practical knowledge which is useful for action and more generic theoretical knowledge. Many contemporary researchers will probably recognize themselves in this description. This encompasses for example research on participatory processes, poverty, gender, business processes, environmental management or policy-making.
Various terms have been coined in the literature to designate such complex systems research. Cundill et al. (2005), for example, call for “integrated approaches”, Audouin et al. (2013) for a self-reflexive, post-reductionist position, and McGowan et al. (2014) for a “more integrative style of inquiry”. A commonality among these authors is that they all emphasize the need for complex systems researchers to draw from various research approaches and use mixed methods, combining elements of qualitative and quantitative viewpoints, data collection, analysis, and inference techniques (Johnson & Onwuegbuzie, 2004). As highlighted by Rosen (1987, in McGowan et al., 2014, p.3) “no one method is sufficient to appreciate the dynamic, emergent, multi-scale behaviour of complex systems”. In this paper, we will thus focus on complex systems research which is engaged, applied, and using mixed methods.
We will adopt in this paper the concept developed by McGowan et al. (2014, p.1) of a “research journey”: “the research journey underlines the breadth of options researchers follow to appreciate and engage with that complexity […]; researchers knowingly move across a “landscape” where different research methods are located”. However, the breadth of the landscape, characterized by the number of existing research approaches and methods which complex systems researchers may draw from, is immense. Research approaches range from case study analyses (Yin, 2009) to grounded theory (Corbin & Strauss, 2015) or action research (Checkland & Holwell, 1998). With such choice available, early career researchers, multi-disciplinary scholars new to complex systems research and doctoral students in particular, may find it difficult to undertake the research journey. They can “quickly become confused and frustrated by the many directions in which their analyses are pulled” (Cundill et al., 2005, p.2). They often ask questions such as: is it possible to mobilize several research approaches while maintaining the coherence and validity of the research? Are some approaches incompatible with one another? Are there specific methods associated with each of these approaches? There is an overall lack of guidance in adopting integrated approaches (Cundill et al., 2005).
Some authors have, nevertheless, started underlining skills and awareness which complex systems researchers need to acquire in order to be able to undertake the research journey, and to study complex social-ecological systems. Notably, they need to engage with various knowledge types and forms (Berkes et al., 2003), including academic and practical knowledge (McGowan et al., 2014). They also need an ability to elicit, and an awareness of, hidden assumptions and values that influence the research process (Audouin et al., 2013). Researchers need to make their research approach and their relative position in the field explicit and transparent (Audouin et al., 2013; Rogers et al., 2013). These skills and awareness are expected to help complex systems researchers to craft their own research approach with confidence, more easily communicate findings and facilitate cooperation among researchers with similar interests and different skill sets. Ultimately, this would facilitate the study of complex social-ecological systems.
The objective of this paper is to help complex systems researchers to gain these skills and awareness to undertake the research journey with more confidence. Specifically, this paper provides a typology of research approaches which can guide complex systems researchers, especially those beginning their research career, to:
 Develop reflexivity and clarify the key assumptions underlying their research,
 Check the coherence of these assumptions, and
 Situate their research in the array of existing research approaches to identify which approaches bear similar or contradictory philosophical assumptions to their own, thus hopefully helping them undertake more informed reading and use of the literature.
In order to meet this objective, this article starts by an overview of the theoretical debates on the use of mixed paradigms, methodologies and methods. This overview highlights the need for a typology of research approaches for complex systems researchers. We then clarify the definitions of key terms that are used in this paper. The following section presents a review of relevant existing categorizations of research approaches. We then present an index of existing research approaches and explain the selection process to identify the seven which are included in our typology. The next section presents a typology based on six key assumptions. The seven research approaches are classified along these assumptions. In the penultimate section an example is provided illustrating the use of the typology for a specific research. The conclusion highlights the added-value and limits of the typology and emphasizes aspects of coherence, reflexivity and transparency in complex systems research.

READ  Monetary macroeconomic theory

Existing theoretical debates and rationale for a typology

Adopting different methods and research approaches raises a number of problems. These problems mainly relate to the fact that different methods or approaches are underpinned by different assumptions. In other terms, they belong to different paradigms. Paradigms can be broadly defined as particular combinations of assumptions (Mingers, 2003, p.559). The term was first defined in relation to scientific research by Kuhn (1962) and further expanded by Burrell and Morgan (1979) who defined it as groupings of theoretical approaches with similar onto-epistemological foundations.
Mingers and Brocklesby (1997) have identified three main problems which researchers may face when adopting a pluralist position. The philosophical problem relates to the fact that such position may not be theoretically coherent because different methods embody the contradictory assumptions of different paradigms. The cultural problem relates to the constraints imposed by academic communities to employ methodologies or methods which correspond to their particular values or beliefs. The psychological problem relates to the fact that researchers have psychologically ingrained preferences and that moving from one paradigm to another or working in a paradigm that calls for actions and behaviours that do not “fit” their preferences may create discomfort and require too much intellectual effort. In addition to these philosophical, cultural and psychological problems, there is a terminology problem. In complex systems research many terms are open to various interpretations and/or tend to be used interchangeably (e.g. research approach, paradigms, methodologies) creating confusion among complex systems research scholars (Grix, 2002). The terminology used in this paper is presented in the next section.

Table of contents :

Introduction
Rationale of this thesis
Summary of literature review and research gaps
Research questions
Clarification of the main terms used in this thesis
Thesis structure and content
Case studies
Assumptions and scope
PART 1 • MONITORING AND EVALUATION
Chapter 1 • Undertaking the research journey: a typology of approaches for complex systems research
1.1 Introduction
1.2 Existing theoretical debates and rationale for a typology
1.3 Terminology used in this paper
1.4 Review of past categorizations of research approaches
1.5 Index of existing complex systems research approaches and identification of relevant ones to be included in our typology
1.6 Towards a typology of complex systems research approaches
1.7 Example of the use of the typology for situating a doctoral research project
1.8 Discussion and conclusion
Chapter 2 • Towards understanding participatory processes: framework, application and results 
2.1 Introduction
2.2 Framework for describing, diagnosing and comparing participatory processes
2.3 Monitoring and evaluation
2.4 Case study application results
2.5 Discussion
2.6 Conclusion
Annex 2.1 • Framework application template
Résumé 12
Chapter 3 • The MEPPP framework: a framework for monitoring and evaluating participatory planning processes
3.1 Introduction: challenges for the monitoring and evaluation of participatory planning processes
3.2 Existing literature related to the M&E of participatory planning processes
3.3 Methods: introduction to the Monitoring and Evaluation of Participatory Planning Processes (MEPPP) framework
3.4 Results and application of the MEPPP framework on a specific case: the Rwenzori participatory planning process in Uganda
3.5 Discussion: strengths and weaknesses of the MEPPP framework
3.6 Conclusion
Chapter 4 • Four challenges in selecting and implementing methods to monitor and evaluate participatory processes: example from the Rwenzori region, Uganda
4.1 Introduction
4.2 A quadruple challenge for M&E methods
4.3 Example of the M&E of the Rwenzori participatory planning process in Uganda
4.4 Discussion
4.5 Conclusion
Annex 4.1 • List of questions used for the interviews
Annex 4.2 • “Rapporteur debriefing sheet” used to guide evaluators in their participant
Annex 4.3 • “Facilitator debriefing sheet” used to guide facilitators in their participant
Annex 4.4 • Thorough questionnaires distributed at the end of each meso workshop
Annex 4.5 • Simple questionnaires distributed at the end of each local workshop
PART 2 • INSTITUTIONAL DYNAMICS
Chapter 5 • A participatory planning process as an arena for facilitating institutional bricolage: example from the Rwenzori region, Uganda
5.1 Three approaches to institutions: institutional design, fit and bricolage
5.2 Main concepts
5.3 Research question, posture and methods
5.4 Narrative of the Rwenzori case
5.5 Discussion and ideas to be experimented by practitioners
5.6 Conclusion
Chapter 6 • Drivers of environmental institutional dynamics in decentralized African countries
6.1 Introduction
6.2 Clarification of the main terms used in this paper
6.3 Methodology
6.4 Introduction to the two cases
6.5 Results: drivers of institutional dynamics
6.6 Discussion
6.7 Conclusion
Chapter 7 • Operationalizing multi-scale INRM in Africa: comparison of regional participatory planning processes in Ethiopia and Uganda
7.1 Introduction
7.2 The suggested approach: regional participatory planning processes
7.3 The Fogera case, Ethiopia
7.4 The Rwenzori case, Uganda
7.5 Case comparison and discussion
7.6 Conclusion
Chapter 8 • Practicing relational leadership through managing frame diversity: example from a participatory process in Ethiopia
8.1 Introduction
8.2 Identifying frames
8.3 Managing frames
8.4 The Fogera participatory planning process case
8.5 Discussion: factors enabling and constraining the effectiveness of strategies to manage frame diversity
8.6 Conclusion
Conclusion
Key contributions from the thesis chapters
Addressing the research questions
Additions to knowledge
Shortcomings of this thesis and agenda for future research
Concluding comment

GET THE COMPLETE PROJECT

Related Posts