Key concepts utilised in IS success evaluation

Get Complete Project Material File(s) Now! »

CHAPTER 3. COMPONENTS OF A METHODOLOGY

Introduction

The aim of this chapter is to present a framework for the development of a methodology suitable for the evaluation of management information systems (MISs) at public Technical and Vocational Education and Training (TVET) Colleges in South Africa. Thus, the first sub-research question in support of the main research question: What are the components of a methodology? (cf. section 1.2.3.2) is addressed in this chapter. The literature review presented herein therefore focuses on definitions and descriptions of concepts, constructs and components within the knowledge domain of methods, methodologies and paradigms.
The chapter is structured as follows: in section 3.2 specific concepts are defined and explicated; findings on a literature review on evaluation methodologies are presented in section 3.3; and in section 3.4 examples of methodologies in the information systems (IS) discipline found in the literature are discussed. Thus the information presented in this chapter should be regarded as the first step in the development of a methodology (artefact) for the evaluation of Management Information Systems (MISs) deployed at public Technical and Vocational Education and Training (TVET) Colleges in South Africa by searching for the building blocks or components of a methodology and the relevant underpinning theories.
The investigation covers one of the aspects within Phase 3 of the research process of the study described in section 1.3.4 namely: define objectives of a solution, which is the second activity in the DSRP model (Peffers et al., 2006, p. 93) as shaded in Figure 3.1.
In Figure 3.2, the part of the framework for the systematic literature review (which was introduced in section 1.3.1) relevant to this chapter is highlighted. The literature overview for this chapter started with keyword searches on: methodology; evaluation methodology; methodology component and methodology artefact. The systematic process through which literature sources were retrieved employed the ABI/INFORM™ complete database provided by ProQuest. Subsequent to keyword searches, more literature resources were found by using literature review techniques as defined and explained in Table 1.1. Thus, literature searches initially started with high level main keywords, but more searches on keywords and references found in the initially collected sources were conducted afterwards in a snowballing manner
Table 3.1 presents the results of the systematic literature overview for the purpose of indicating scope. Since the term methodology is commonly used to indicate the set of methods utilised to conduct a research study, it was decided to launch the main keyword searches only on titles of literature sources. Relevant literature was selected for further scrutiny by reading through the full titles of the resources, and if believed relevant, the abstracts were evaluated for inclusion.
The following section provides definitions of key concepts utilised in this knowledge domain.

Terms and definitions

McGregor and Murnane (2010) acknowledge that the three interrelated concepts: method, methodology and paradigm are inconsistently used in the literature. Many scholars do not clearly differentiate between these concepts and in many instances they are incorrectly used interchangeably. The following sections elucidate the distinct difference between these concepts. In this section the meaning of the terms: method, methodology and paradigm are clarified.

Method

The American Heritage Dictionary (2009), accessed online, defines the term method as:

• “A means or manner of procedure, especially a regular and systematic way of accomplishing something;
• orderly arrangement of parts or steps to accomplish an end;
• the procedures and techniques characteristic of a particular discipline or field of knowledge” (Houghton Mifflin Company, 2009).

Goldkuhl and Lind (2010, p. 46) define a method as a set of steps (an algorithm or guideline) to perform a task. Research methods are the tools, techniques or processes used during a scientific investigation and can, for example, include surveys, interviews, photovoice or participant observation. Methods are characterised as either qualitative or quantitative (Kinash, 2006, p. 6).
In a design science research paradigm a method (artefact) defines processes. A method provides guidance on solving a problem, by searching through the “solution space” (Hevner et al., 2004, p. 79). These methods (artefacts) can range from formal mathematical algorithms that explicitly define the search process to informal, textual descriptions of best practice approaches, or some combination of the two (Hevner et al., 2004, p. 79).

Methodology

The American Heritage Dictionary (2009), accessed online, defines the term: methodology as follows:

• “A body of practices, procedures, and rules used by those who work in a discipline or engage in an inquiry;
• a set of working methods including the study or theoretical analysis of such working methods;
• The branch of logic that deals with the general principles of the formation of knowledge” (Houghton Mifflin Company, 2009).
• Irny and Rose (2005) explain that a methodology is generally a guideline for solving a problem and it includes components such as phases, tasks, methods, techniques and tools. Robson (1997) furthermore suggests that methodologies generally encompass the following four elements:
• provide an opinion of what needs to be solved;
• define techniques on what has to be done and when it should be done;
• advise on how to manage the quality of deliverables or products; and
• provide a toolkit to facilitate the process (Robson, 1997).

Arbnor and Bjerke (2009, p. 3) argue that a methodology is a mode of thinking, but also a mode of acting. It contains a number of concepts which try to describe the steps and relations needed in the process of creating and searching for new knowledge to solve problems (Arbnor & Bjerke, 2009, p. 3). A methodology is also described as a system of broad principles or rules (that guide research practices) from which specific methods or procedures may be derived to interpret or solve different problems within the scope of a particular discipline. A methodology is not a formula like an algorithm, it is a set of practices (WebFinance, 2015) which is guided by a set of principles and a common philosophy for solving targeted problems (Checkland, 1981). In the context of research “a discipline’s methodology, broadly defined, is its body of methods, approaches, rules, and techniques” (Weiss, 2005, p. 1).
Mingers (2001) synthesises various authors’ definitions of a methodology and states that the term methodology, could be interpreted in three different ways. Firstly, it could refer to the study of methods; secondly, it could refer to a methodology of a specific study; and thirdly, it could refer to a generalisation of a specific methodology. Explanations of the three types are provided in the following sections.

Study of methods

In this context, the term ‘method-ology’ means the study of methods (Checkland, 1981; Mingers, 2001). Here one would refer to a programme or course in Research Methodology which includes a range of different methods. In this context, a methodology is a systematic, theoretical analysis of the methods applied to a field of study and does not set out to provide solutions (Irny & Rose, 2005, p. 330). Methods are categorised as either quantitative or qualitative. A qualitative methodology includes a set of qualitative methods; similarly, a quantitative methodology includes a set of quantitative methods and a mixed-methods methodology includes both quantitative and qualitative methods.
 

Methodology of a specific study

In this instance the term refers to a specific set of research methods utilised in a particular research project, or study, to provide or suggest a solution to a specific problem. Every research study has its own individual methodology (Mingers, 2001). Irny and Rose (2005) categorise this type of methodology as specific-to-context methodologies. From a practical business perspective, this type of methodology, as developed and applied within an organisation, is “… all about cementing credibility in our practice and consistency in our approach”, as Jeffrey McIntire mentioned in his 2010 Web Content talk (Mcintire, 2010).
 

Generalisation of a specific methodology

In practice, particular combinations of methods are used many times or are deliberately designed a priori. By using the term methodology in this manner, a more general and less prescriptive process than a method is implied (Irny & Rose, 2005). According to Mingers “it is a structured set of guidelines or activities to assist in generating valid and reliable research results. It will often consist of various methods or techniques, not all of which need to be used every time” (2001, p. 242). Mingers (2001) adds that it can be difficult to precisely delineate the boundaries between method and methodology at one end (e.g. administering and analysing a survey), or between methodology and a general research approach (e.g. qualitative research methodology) at the other. A methodology could therefore be concerned with combining research methods, but it is also possible to combine these more generic methodologies.
It is envisaged that the methodology (artefact) developed in this study will fit into this category.
The meaning of the third concept: paradigm is presented in the following section.

Paradigm

The American Heritage Dictionary (2009), accessed online, defines the term: paradigm as a “set of assumptions, concepts, values, and practices that constitutes a way of viewing reality for the community that shares them” (Houghton Mifflin Company, 2009). Kuhn (1970) explains that a paradigm includes an accepted body of rules and techniques for solving problems within a specific community of practice and that research is guided by paradigms through direct modelling as well as abstracted rules.
McGregor and Murnane (2010, p. 420) define the term paradigm to encompass two dimensions namely philosophical: basic beliefs and assumptions about the world, and technical: the methods and techniques adopted when conducting research. The authors furthermore explain that a methodology represents the philosophical underpinning of a paradigm, and its methods and how they are used are shaped by this methodology. Hence, a specific methodology explains why certain methods or tools are used. A methodology offers the theoretical underpinning for understanding which method, set of methods, or so-called best practices can be applied to a specific case (Kuhn, 1970).

Delineation of the term methodology as used in this study

The methodology (artefact) developed in this study falls in the category as described in section 3.2.2.3. The developed artefact will be a generalised methodology (artefact) that can be applied in the domain of success evaluation of MIS at public TVET Colleges.
In addition, it is important to distinguish between the different connotations of the term methodology as used in this study. The term methodology was used in four contexts namely

• The final product of this study was an artefact called a methodology for the evaluation of the success of MISs deployed at public TVET Colleges. To enable distinctive identification of this methodology, it will henceforth be referred to as the TVET-MIS-EVAL methodology.
• Another context in which the term methodology was utilised is located in Chapter 2, in which the specific research design and methodology of this study are explicated.
• Additionally, within the underpinning philosophical paradigm of the study, which is design science (DS), the design science research methodology (DSRM) with its design science research process (DSRP) model was also referred to in this study. The DSRP model informed the research process of the study.
• Finally, in Chapter 7, a methodology was used for the selection of the sample of cases on which the artefact (TVET-MIS-EVAL methodology) was demonstrated and evaluated. This methodology entailed the development of a questionnaire based on web maturity models theory and the clustering of the population of public TVET Colleges.

The following section provides information regarding the theory that underpins the context (methodological views) in which methodologies can be utilised.

Methodological views

Arbnor and Bjerke (2009, p. 7) identify different methodological views based on when and how these methods are used for studying and researching reality. The authors argue that a methodological view is its concepts. Methodological views make ultimate presumptions (from background information) about reality. The authors see methodologies as guiding principles for the creation of knowledge. In order to be useful and effective, these principles must fit both the problem under consideration and the ultimate presumption (Arbnor & Bjerke, 2009, p. 11). The authors furthermore describe three methodological views namely: the analytical view, the system’s view and the actor’s view (Arbnor & Bjerke, 2009, p. 19).
A methodology in the analytical view explains causality in reality where said reality is presumed to be based on facts. There exist subjective facts and objective facts. Concepts of the analytical view include: reality and models, deduction, hypotheses, causal relations, induction and verification and analyses (Arbnor & Bjerke, 2009).
A methodology in the system’s view explains and understands reality, presumed to be built up holistically. The systems view includes three overlapping philosophies namely: systems theory, holism and structuralism. Systems theory is the interdisciplinary study of organisations including systems language and thinking. Structuralism includes theories across social sciences and humanities which assume that much can be learnt from structured relationships. Holism propagates the idea that a system under investigation cannot be explained, understood or determined by its components alone. It is believed that the system determines how the components behave (Arbnor & Bjerke, 2009).

DEDICATION 
ACKNOWLEDGEMENT 
DECLARATION 
ABSTRACT
CHAPTER 1. INTRODUCTION 
1.1. Background to the study
1.2. Problem statement and purpose
1.3. Theoretical grounding of the research
1.4. Importance and relevance of the study
1.5. Contribution
1.6. Delineations, limitations and assumptions
1.7. Future research
1.8. Research planning .
1.9. Summary
1.10. Chapter map for the thesis
CHAPTER 2. RESEARCH DESIGN AND METHODOLOGY 
2.1. Introduction
2.2. Philosophical assumptions of the study
2.3. Philosophical paradigms.
2.4. Theoretical background of the study
2.5. Process model of the study adapted from the DSRP model
2.6. Data collection methods
2.7. Data analysis methods
2.8. Ethical considerations
2.9. Conclusion
CHAPTER 3. COMPONENTS OF A METHODOLOGY 
3.1. Introduction .
3.2. Terms and definitions
3.3. Evaluation methodology
3.4. Exemplars of methodologies in information systems research
3.5. Conclusion – Components of a methodology
CHAPTER 4. MODELS FOR THE EVALUATION OF MANAGEMENT INFORMATION SYSTEMS 
4.1. Introduction
4.2. Situation of the literature review
4.3. Background to Information Systems evaluation
4.4. Key concepts utilised in IS success evaluation
4.5. Theories underpinning IS success models
4.6. IS success models
4.7. Motivation for the selected base model
4.8. Model for the TVET-MIS-EVAL methodology
4.9. Constructs and effectiveness measures
4.10. Instrument development
4.11. Conclusion
CHAPTER 5. PUBLIC TVET COLLEGES AND MISS DEPLOYED 
5.1. Introduction
5.2. Situation of the literature review
5.3. Background to the TVET sector
5.4. Developments in colleges’ data reporting systems
5.5. Conclusion
CHAPTER 6. TVET-MIS-EVAL METHODOLOGY (ARTEFACT) 
6.1. Introduction
6.2. Philosophical assumptions of the artefact
6.3. The TVET-MIS-EVAL methodology
6.4. Conclusion
CHAPTER 7. CLUSTERING OF PUBLIC TECHNICAL AND VOCATIONAL EDUCATION AND TRAINING (TVET) COLLEGES AND SAMPLE SELECTION 
7.1. Introduction
7.2. Literature Review
7.3. Research Design
7.4. Results and Findings
7.5. Discussion
7.6. Conclusion
CHAPTER 8. ARTEFACT DEMONSTRATION AND EVALUATION THROUGH CASE STUDIES TOWARDS REFINEMENT OF THE TVET-MIS-EVAL METHODOLOGY 
8.1. Introduction
8.2. Application of the artefact (TVET-MIS-EVAL methodology)
8.3. Characteristics of the selected colleges
8.4. College 1: Findings transpired from the application of Phase C: Procedures and Phase D: Toolkit of the TVET-MIS-EVAL methodology
8.5. College 2: Findings transpired from the application of Phase C: Procedures and Phase D: Toolkit of the TVET-MIS-EVAL methodology
8.6. College 3: Findings transpired from the application of Phase C: Procedures and Phase D: Toolkit of the TVET-MIS-EVAL methodology
8.7. Summary of findings on the three cases.
8.8. Suggestions for refinements to the TVET-MIS-EVAL methodology (Artefact)
CHAPTER 9. SYNTHESIS OF FINDINGS, DISCUSSION, CONCLUSION, LIMITATIONS AND RECOMMENDATIONS 
9.1. Introduction
9.2. Research overview
9.3. Thesis questions answered
9.4. Summary of the research design
9.5. Reflection on key findings
9.6. Chapter summary
9.7. Contribution to knowledge
9.8. Delineation and assumptions of the research study
9.9. Three reflections on the study
9.10. The way forward
9.11. Conclusion
REFERENCES
GET THE COMPLETE PROJECT
METHODOLOGY FOR THE EVALUATION OF MANAGEMENT INFORMATION SYSTEMS AT PUBLIC TECHNICAL AND VOCATIONAL EDUCATION AND TRAINING COLLEGES IN SOUTH AFRICA