THE LOGIC MODEL AS A GUIDE FOR THE PROCESS EVALUATION OF SSIP

Get Complete Project Material File(s) Now! »

CHAPTER 3 THE THEORETICAL FRAMEWORK THAT UNDERLIES PROCESS EVALUATION OF THE SECONDARY SCHOOLS INTERVENTION PROGRAMME

INTRODUCTION

The purpose of theoretical framework and the theories that influenced the study under investigation are discussed in this chapter. A theoretical framework is required in each study in order to guide policymakers and programme designers in understanding how the intervention was evaluated. The framework, which serves as a foundation for the evaluation of the intervention programme under investigation, is presented. The subsequent discussion highlights the relationship and the influence of the concepts used in the study in relation to the development of the intervention framework for SSIP. In the section below the researcher provides an explanation for using the theoretical framework for evaluating the intervention programme.

THE PURPOSE OF A THEORETICAL FRAMEWORK

Evaluation like any other research is guided by a theoretical framework. A theoretical framework provides a structure to help support a theory of the evaluation research study (Swanson, 2013). In other words a theoretical framework is needed to guide the evaluation of the intervention programme under study and gives meaning to the gathered information. Using a theoretical framework assures the reader that evaluation is informed by established theory and empirical facts obtained from credible studies. It helps the reader to understand the perspective of the researcher (Trochim, 2006). Furthermore, the theoretical framework provides the researcher with a lens to view the world from different perspectives (Merriam, 2001; Mackenzie and Knipe, 2006; Troudi, 2010). This helps the reader to understand the researcher’s perspective before making any form of judgment about the study.
In the case of the SSIP, the theoretical perspective guides the study and determines its focus. The focus of the study influences the selection of an appropriate methodology to conduct the evaluation study, that is, how information should be obtained and how it should be processed and summarised (McMillan and Schumacher, 2010:322). This means the theoretical framework influences the researchers’ method of investigation. It guides research, determines what things to measure and/or what statistical relationships to look for. In designing the theoretical framework, the researcher aims at helping the reader to make logical sense of the relationships of the variables and factors that are considered relevant to the problem.
Prior to discussing the theoretical framework used in this study, it is important to focus on the theories that influenced and guided the evaluation of the intervention under investigation. The discussion below guided the researcher to select the research methodology used to evaluate the SSIP programme.

THE INTERPRETIVIST THEORY IN PROCESS EVALUATION

The theoretical framework for the intervention programme focusing on secondary schools used in this study is based on the philosophy of interpretivism. Interpretivism assumes that the reality in evaluation is socially constructed (Given and Saumure, 2008; Yilmaz, 2008). This means the meaning and understanding of evaluating the phenomenon under study is socially produced by information sourced from individuals participating in the programme. In order to understand the phenomena under study the interaction between participants and the researcher (Babaheidari, 2007) plays a significant role.
Interpretivism arose from scholars such as William Dilthey, Edmund Husserl and Weber who argued that human sciences (geisteswissenschaft) were in essence, different from the natural sciences (naturwissenschaft) because science is based on abstract explanations while human sciences is based on understanding the everyday lived experiences of people in their specific contexts (de Vos et al., 2012). These scholars criticised science for refusing to acknowledge that theories and realities are not waiting somewhere to be discovered but emphasised that all knowledge, including evaluation theories and methodologies, are constructed by humans.
Scotland (2012) concurs with them and adds that meaning is not discovered but constructed by individuals interacting with self-consciousness and the world. Such constructions are created by individuals or groups of people through their discussions and various forms of interactions. Thus, interpretivists believe that reality is constructed through social interactions and how people perceive it (Jones and Hughes, 2001; Wahyuni, 2012).
The lifeblood of interpretivism is the understanding of subjective meaning of which the meaningful reality is constructed through interaction between individuals and their environment and transmitted in a social context. In order to extract information and understand the phenomena under study, it is imperative to explore ways in which people make sense of their experiences and the environment. Babbie and Mouton (2009) adds that all human beings are engaged in the process of making sense of their worlds and continuously interpret, create, give meaning, define, justify, and rationalise daily actions. In order to understand the subjective and social reality of human beings, an interpretivist need to have actually comprehended the meaning(s) of the actions presented as well as the context in which evaluation is taking place. To be able to comprehend the action presented it is required of the evaluator to be able to interpret the observed actions within context.
As interpretivism is regarded as a subjective approach to evaluation, researchers such as Garrick (1999); Douthwaite, Kuby, van de Fliert and Schultz (2002); Rowland (2005); de Vos et al. (2012) and Scotland (2012) assert that evaluation research should be qualitative. An interpretive approach to evaluation research furthermore relies heavily on inputs by participants (Cuthill, 2000; Creswell, 2009; Thomas, 2010) and as such qualitative methods of data collection would seem most appropriate. Mackenzie and Knipe (2006) indicate that in some cases it might be necessary to use mixed methods research in evaluation studies but this study employed a qualitative approach.
Using an interpretive paradigm and qualitative approach to evaluate SSIP, the researcher needed to search for information from participants’ and gain insightful knowledge regarding the programme by taking advantage of their (participants’) personal teaching and learning experiences. This process provided the researcher the opportunity to interact and communicate with various stakeholders and/or participants as a way of constructing new knowledge. In addition creating the link between the needs and desires of participants and the anticipated outcomes as planned by management provided valuable and meaningful information. The social interaction that developed between participants and the researcher was helpful in that the researcher learned to understand what is happening as participants related to and reflected on their experiences. As the researcher listened to participants’ reflection on their lived experiences, attention was placed on understanding and making sense from the stories they related in order to interpret those experiences.
Developing an understanding of meaning in evaluation is not easy (Bell and Aggleton, 2016), as it requires openness and dialogue between different stakeholders. Thus, meaning is understood through open discussions and negotiation with various stakeholders. This kind of activity is necessary as stakeholders involved in the programme are identified on the bases of their insightfulness and it is assumed that they will provide valuable information about the programme and might contribute in suggesting ways in which the programme can be improved.
Individual participants involved in process evaluation of the programme are afforded the opportunity to reflect on their experiences as the programme is being delivered. This is one way of monitoring the programme and establishing whether the programme is being delivered as planned. The information gathered, analysed and interpreted will then help programme designers, policymakers and managers of the programme to shape and improve it. The information provided will guide key stakeholders as to what is good and relevant about the programme and what needs to be removed or abandoned, if necessary.
Methodologically, the interpretive paradigm prefers data to be obtained from the natural setting in order to make sense of the programme. The researcher or evaluator in this regard is required to acquire information by studying or observing the subject while in action, in the natural environment (Gravetter and Forzano, 2016). In other words, the researcher observe the participants while in action and watch what is happening to acquire first-hand information in the natural setting. Due to the fact that the researcher employs direct observation of participants and also interact with stakeholders involved in the programme to source information and learn from their experience, he/she is tries to understand their experiences, learn in order to make sense of their world. This suggests that evaluation from the interpretivist point of view is based on qualitative methods and it is context-based, which may connect with Theory of Change (ToC).
The next section focusses on understanding ToC.

UNDERSTANDING THE THEORY OF CHANGE

The theory that underpins this study is a Theory of Change (ToC). ToC is a strategy that is proposed by Carol Weiss (1972) and it developed from Theory-Based Evaluation (TBE). Theory-Based Evaluation (TBE) has been given different labels by various researchers who utilise different terms interchangeably to refer to the same thing. It is referred to it as theory driven evaluation, programme theory, theories of practice, programme theory, theory of change, theory driven evaluation, evaluation theory and/or the logic model (Chen, 2005b, 2005c; Donaldson and Lipsey, 2006; Donaldson, 2007; Hansen and Vedung, 2010. It seems to depend on whom and how one plans to utilise the theory to investigate the phenomenon under study.
Research suggests that a Theory of Change can be used in three different ways. It can be used as discourse or as a tool or as an approach (Valters, 2015). As a discourse, it forces one to interrogate one’s assumption about change and asking questions like “What is my/your ToC? In other words one is forced to self-examine one’s beliefs about change and what it means. Such an examination requires one to reflect and explore one’s beliefs about change and how it happens. ToC is one way of reflecting and thinking about change and how it happened over the years. Reflection is an important aspect of learning; as John Dewey (1933) says we do not learn from experience but we learn from reflecting on experience acquired.
The acquired experience in peoples’ lives helps them to reflect, learn and work on the change one wants to institute. ToC can play an important role in the process of reflection to explore the change one intends to achieve. James (2011) says that a ToC is an ongoing process of reflection to explore change, how it happens and what it means for the part we play in a particular context, sector and/or group of people. Process evaluation is one way of monitoring the process of change and reflecting on how learners are in the process of improving their performance to change for the better.
As a tool a ToC is often used as a way of making the assumptions connecting the activities, outputs and outcomes clearer and making them more explicit (Valters, 2015). Assumptions are used to provide an explanation of the connections between the preconditions for long-term changes that occurs in early and intermediate stages of the change process and an expectation of how and why the proposed interventions will bring about change (Anderson, 2004). Weiss (2000, 2004) maintains that to design and evaluate an intervention, the assumptions of the envisaged programme must be clearly defined. This tool helps users of a ToC to create a framework and a mode of change that maps out how the programme plans on getting from a particular point to the end of the programme (INSP, 2009). For the process evaluation in this study, the researcher believes that by unpacking the assumptions about the programme, stakeholders involved in the programme will understand the relationship between the problems they are addressing and the strategies they are planning to use in order to get the work done (Mackinnon, Amott and McGarvey, 2006) to improve the programme in such a way that it is eventually delivered as intended.
As an approach a ToC develops the thinking and practice of what is known about how the organisation can make an effective contribution to the social change in a complex environment (Stein and Valters, 2012). This means a ToC occupies a broader space than a tool and as such it should be seen as an inclusive tool. For example, the group involved in the change process is afforded the opportunity to think critically about what is required to bring about a desired social change (Anderson, 2004). In the case of learners, they think of strategies that than help leaners improve their situation for themselves to perform better. Using a ToC as an approach helps stakeholders to understand how a complex change process will unfold over time and acknowledge that change is a process and not an event (Fullan, 2003). As such it needs time and commitment to be realised.
James (2011) argues that a ToC used as an approach involves an analysis at an organisational level and link processes like outcome mapping, log frames and associated tools at project level to describe what one plans to do and help review progress at that level. A ToC is not selective as it can be used and be applicable at different levels of the organisation on macro- or micro-levels. It provides users with the opportunity to describe their plan at each level because each level might have a different interest regarding change. For example, the contribution at macro-level can be made in terms of mission, vision, beliefs, capacity and approach (James, 2011). Then, they need to explain how they envisage change to happen at that level. A ToC approach at micro level will be different as the needs differ from those at macro level.
It should be noted that using a ToC is an intense and time-consuming process. The stakeholders’ interaction and their contribution made through their reflection, dialogues, critical thinking and analysis of issues from different perspectives helps them to develop a plausible, doable and testable framework (Weiss, 2004; Vogel, 2012). According to various researchers, a plausible framework provides evidence that the suggested activities will lead to the desired outcomes; what is doable indicates the availability of resources to carry out the initiative and test them (Sunal and Wright, 2006; Waltz, Strickland and Lenz, 2010:463; Chen, 2012). The evaluator is allowed to track the progress the programme is making in a credible and useful manner. These attributes must be identified before any commitment to evaluate the programme is made.

Weiss’s articulation of assumptions

According to Weiss (1997:265) a ToC refers to a chain of assumptions that explains how the programme activities lead to step by step achievement of the desired outcomes. They are also statements and beliefs that guide the rationale behind the programme (Weiss, 1997; 2000; 2004). Weiss (2000, 2004) indicates that to design and evaluate the intervention theory, the assumptions of the envisaged programme must be clearly defined. In other words, the study of any evaluation should state exactly what these assumptions are and how the programme will bring about change.
It is important to define and clarify assumptions because the explicit articulating of assumptions helps in evaluating programmes, no matter how complex they might seem. Ignoring this process makes it difficult to evaluate any form of programme (Weiss, 2004). The process helps stakeholders to guard against using faulty assumptions, which may lead to poor result findings. Therefore, clarifying the underlying assumptions of the initiative will have to be detailed in such a way that the theory can be tested and measured (Connell and Klem, 2000; Hansen and Vedung, 2010; Stein and Valters, 2012). Assumptions are important aspects of the programme and spelling out the assumptions is critical as this makes it easier to evaluate the programme.
Weiss (2005) introduced a sequence of steps that demonstrate how the expected outcomes of an intervention can be realised. She put into place an evaluation strategy to track the steps and activities that build towards the outputs and outcomes to determine whether those expected outcomes are actually produced through those steps. Otherwise, if the clarification of the mini-steps are lacking, the sequences that must be taken to reach a long-term goal reduces the possibility that critical factors related to the outcomes will be addressed. Clarifying assumptions is a significant step to help key stakeholders to identify the beliefs they have about the programme and think through how the programme will operate. A ToC is in reality meant to help clarify and simplify people’s thinking and understanding of a project, its implementation and the delivery process of the programme. It is important to employ a ToC strategy at each stage of programme evaluation.
The section below discusses reasons for selecting a ToC for guiding the evaluation of the SSIP programme.

Rationale for Selecting Theory of Change

Selecting a ToC as a base for process evaluation of SSIP is beneficial as it would make stakeholders and those involved in the programme interested in understanding how the activities are used to help programme participants achieve the planned outcomes of the programme. Denying them (stakeholders) exposure and excluding them from contributing to the process will harm the entire evaluation process rather than affording them the opportunity to take part in and own the process. The researcher concurs with Weiss (2005) that involving stakeholders encourages engagement on matters that would serve the needs of the participants (in this case the district and under performing schools). In this case stakeholders will understand how the performance of learners could improve because of the SSIP.
Another important reason for selecting a ToC is that the process is open, transparent and allows all those involved to learn and understand the connection of the processes from one step to the next and how change comes about. Furthermore, it is practical, realistic and the undertaking stimulates debates among the community of participants. The community of participants in this case would mean key stakeholders, such as decision makers, programme managers, coordinators as well as programme facilitators. The varied and differing views, experiences and knowledge that participants bring to the table is very important. This brings a diversity of wisdom as participants listen and learn from each other. This collaborative and collective effort helps in creating a meaningful strategy and provides direction towards improvement of learners’ performance, which may be seen as a key to success.
The section below discusses the theoretical framework for SSIP.

THEORETICAL FRAMEWORK GUIDING PROCESS EVALUATION OF SSIP

The theoretical framework for the process evaluation of SSIP is based on a ToC refers to the mechanism that mediates between the delivery of the programme and the emergence of the intended outcome that leads to change (Weiss, 1998; Young, Crow and Murphy (2009). A ToC focuses on the internal operations of the programme and examines the links between resources, processes and the results. Understanding a ToC complements process evaluation that focuses on the implementation plan, processes and procedures of programme delivery. According to Hawe et al. (2003) and Patton (2008) process evaluation focuses on the internal dynamics and actual operation of the programme. Identifying those dynamics and the gaps in this study will help the researcher to track how the programme or service being delivered is progressing.
A ToC and process evaluation are sharing similar concerns namely that of understanding how the programme is operating internally and if what is made available could bring about change as planned. While a ToC examines the link between resources and activities wanting to understand how change is brought about, process evaluation is interested at how activities in the programme are being delivered to change the lives of stakeholders participating in the programme. To understand how change takes place Weiss’s (1995) framework does not only specify the programme’s outcomes but how and why of the programme by identifying resources, activities, outputs and outcomes of the programme. The components of investigation are inputs or resources, activities and outputs. The illustration presented below is an example of a ToC mode

DECLARATION 
DEDICATION 
ACKNOWLEDGEMENTS
ABSTRACT 
KEY CONCEPTS 
ABBREVIATIONS
CHAPTER 1 ORIENTATION
1.1 INTRODUCTION
1.2 BACKGROUND TO THE PROCESS EVALUATION RESEARCH
1.3 PROBLEM STATEMENT
1.4 AIMS AND OBJECTIVES OF THE STUDY
1.5 EVALUATION RESEARCH DESIGN AND METHODOLOGY
1.6 CHAPTER DIVISION
1.7 SUMMARY –
CHAPTER 2 UNDERSTANDING PROCESS EVALUATION
2.1 INTRODUCTION
2.2 CONTEXTUALISATION OF EVALUATION
2.3 POLICIES AND PRACTICES RELEVANT TO EVALUATION OF INTERVENTION PROGRAMMES
2.4 FACTORS INFLUENCING EVALUATION CONTEXTS
2.5 INTERVENTION PROGRAMMES
2.6 THE CONTEXT OF SECONDARY SCHOOLS INTERVENTION PROGRAMME
2.7 APPROACHES TO EVALUATION RESEARCH
2.8 PROCESS EVALUATION OF INTERVENTION PROGRAMMES: CONCEPTUAL FRAMEWORK
2.9 EMPIRICAL INVESTIGATIONS OF INTERVENTION PROGRAMMES
2.10 USE OF PROCESS EVALUATION TO EVALUATE INTERVENTION PROGRAMMES
2.11 SUMMARY
CHAPTER 3 THE THEORETICAL FRAMEWORK THAT UNDERLIES PROCESS EVALUATION OF THE SECONDARY SCHOOLS INTERVENTION PROGRAMME
3.1 INTRODUCTION
3.2 THE PURPOSE OF A THEORETICAL FRAMEWORK
3.3 THE INTERPRETIVIST THEORY IN PROCESS EVALUATION
3.4 UNDERSTANDING THE THEORY OF CHANGE
3.5 THEORETICAL FRAMEWORK GUIDING PROCESS EVALUATION OF SSIP
3.7 THE LOGIC MODEL AS A GUIDE FOR THE PROCESS EVALUATION OF SSIP
3.8 THE DIFFERENCE BETWEEN THEORY OF CHANGE AND LOGIC MODEL
3.9 SUMMARY
CHAPTER 4 EVALUATION RESEARCH DESIGN AND METHODOLOGY
4.1 INTRODUCTION
4.2 RATIONALE FOR EMPIRICAL EVALUATION RESEARCH
4.3 PROCESS EVALUATION RESEARCH DESIGN
4.4 EVALUATION RESEARCH METHODOLOGY-
CHAPTER 5 PRESENTATION OF EVALUATION FINDINGS AND ANALYSIS OF DATA
5.1 INTRODUCTION
5.2 BRIEF ANECDOTAL REPORT ON DATA COLLECTION PROCESS
5.3 DATA COLLECTION PROCEDURE
5.4 STAGES OF ANALYSIS OF DATA
5.5 SUMMARY
CHAPTER 6 INTERPRETATION OF DATA AND DISCUSSION
6.1 INTRODUCTION
6.2 INTERPRETATION AND DISCUSSION
6.3 THEME 2: TYPES OF RESOURCES
6.4 THEME 3: THE DELIVERY OF INTERVENTION
6.5 THEME 4: EXPERIENCES OF TUTORS AND LEARNERS IN THE PROGRAMME
6.6 THEME 5: ASSESSMENT
6.7 CONCLUDING REMAKRKS
CHAPTER 7 SUMMARY, CONCLUSIONS AND RECOMMENDATIONS
7.1 INTRODUCTION
7.2 SUMMARY OF PROCESS EVALUATION RESEARCH FINDINGS
7.3 CONCLUSIONS AND RECOMMENDATIONS OF EVALUATION RESEARCH
7.4 RECOMMENDATIONS FOR FURTHER EVALUATION RESEARCH
7.5 LIMITATION OF THE STUDY
7.6 CONCLUDING REMARKS
REFERENCES
GET THE COMPLETE PROJECT
PROCESS EVALUATION OF THE SECONDARY SCHOOLS INTERVENTION PROGRAMME