Computer Technology and Engineering Education

Get Complete Project Material File(s) Now! »

Data Collection & Results


This research study examined the use of a computer-based, immersive, multimedia teaching intervention in the form of a simulated manufacturing organisation or virtual enterprise. This chapter records the data that was obtained over four iterations and five applications of the intervention. It was collected from several sources in accordance with the principles of the design-based research methodology that was adopted for the project and described in Chapter 3. That is, information was gathered from multiple sources including student surveys and questionnaires, group and individual semi-structured interviews with students and the researcher’s observations of students’ interaction with, and reception of, the intervention. The data is used to assist in determining the extent to which the intervention design met its goals (specified in Chapter 1) of improving the delivery of a manufacturing systems course occupying a complex, primarily nonquantitative and ill-structured domain and of providing answers to the research questions. The chapter also includes details of the results of surveys of students’ learning styles and levels of intellectual development utilising appropriate survey instruments as reviewed in Chapter 2, Section 2.7.15, and described in Chapter 3, Section Kolb LSI learning style surveys were carried out to determine if the designed pedagogical approach of the intervention matched the most common student learning style preferences. The Moore LEP survey was applied to determine if the student tasks in the intervention were presented with an appropriate level of indeterminacy, or ‘vagueness’. An appropriate level being one which encourages students to accept, intellectually, that problems may have more than one valid answer whilst at the same time not requiring them to perform too much of a rapid intellectual leap from dualism to relativism. The aim was to encourage students to move from positions of duality to positions of relativity. This issue was discussed in detail in Chapter 2, Section The task formats and presentation must encourage this intellectual development but not be aimed at a relativistic position too far from the typical student’s current position in order to prevent discouragement, and frustration with the ill-defined elements of the tasks.

Data Collection Programme

Several data collection techniques were employed in this research. The use of different methods to collect data allows the researcher to collect multiple perspectives on the design of the intervention and of students’ perception of its utility. The use of varied techniques also assists in contributing to the credibility of the data and is a practice known as triangulation. The methods of data collection are described in detail in Chapter 3, Section 3.4. The data was collected from course participants in five sequential, annual presentations of the Manufacturing Systems course at the University of Auckland. This course is of twelve-week’s duration and is offered in the second semester to undergraduates in the third year of a four-year undergraduate degree in mechanical engineering. The data was collected from a pilot study and four further iterations, as described in Chapter 4, with the instruments described below. Information from each stage was analysed and then used to design and implement improvements to the virtual environment and to the student tasks contained within it. Details of the data collected are set-out in Section 5.3. The data collection programme was as follows: Year one – Pilot Design: A two-part student questionnaire and the researcher’s observations were used to evaluate students’ perceptions of the intervention’s pilot design. The questionnaire was designed to separately evaluate (a) the intervention’s virtual factory interface and (b) the assistance provided by the intervention to the students’ reception of the course topics. A copy of the student questionnaire is included in Appendix E. Year Two – Design Iteration One: Data from group, semi-structured, interviews, conducted with four groups of three students each, combined with the researcher’s observations, were used to assess students’ experiences with, and impressions of, the virtual factory intervention. Year Three – Design Iteration Two: Individual, semi-structured, interviews were conducted with students and, with the researcher’s observations, were used to evaluate the students’ perceptions of a revised intervention design. A copy of the individual interview structure is included in Appendix E. Year Four – Design Iteration Three: The researcher’s observations of students’ reception of, and interaction with, the third iteration of the intervention’s design was used to assess the effects of the major design changes made as a result of feedback from iteration number two. Kolb’s Learning Style Index and Moore’s Learning Environment Preferences instruments were trialled and administered to students to collect data on the distribution of student learning styles and students’ positions on the Perry scale of intellectual development. Copies of these instruments are contained in Appendix E. Year Five – Design Iteration Four: Student questionnaires, individual semi-structured interviews and researcher observations were used to assess students’ reception of iteration number four. The questionnaire, a modified version of the instrument used for the pilot design, was designed to separately evaluate (a) the virtual enterprise’s computer interface and (b) the learning assistance provided by the intervention. A copy of the questionnaire is included in Appendix E and a copy of the interview structure in Appendix E. Kolb’s ‘Learning Style Index’ was completed by students, pre-course and post-course, and Moore’s ‘Learning Environment Preferences’ instrument was also administered.


Data Collection

Data – Pilot Design

Student Questionnaire Structure

The purpose of each question in the Pilot Study questionnaire and its relationship to the research questions, where relevant, are set out in Table 5-1. Students were requested to answer ‘strongly agree’, ‘agree’, ‘undecided’, ‘disagree’ or ‘strongly disagree’ to each question. A copy of the questionnaire is located in Appendix E. From an analysis of Table 5-3 above it can be seen that there was generally a high level of satisfaction with the web site. The highest levels of dissatisfaction identify a pattern which indicates that further effort should be made to improve the clarity of the hyperlinks and the ease with which project information and resources can be found. The ratings of Part B of the questionnaire show a high level of positive support for the proposition that the intervention added interest and relevance to the tasks. The highest negative response (11.8) was to the question of the simulated industry placed scenario adding interest to the task. Although 66.6% answered positively to this question the negative response may be due to the fact that this stage of its development the virtual enterprise had only one task of this nature. The questionnaire response data in Table 5-4 above is shown reformatted in the cluster bar chart below.

1. Introduction
1.1. Background
1.2. Context of the Study
1.3. Problem Statement
1.4. Stages of the Research
1.5. The Structure of the Research Report
1.6. Ethical Considerations, Limitations and Delimitations
1.7. Significance and Benefits of this Research
2. Literature Review
2.1. Introduction
2.2. The Review Process
2.3. Theories of Learning
2.4. Computer Technology and Engineering Education
2.5. Models of Instruction for Virtual Environments
2.6. Factors Influencing Student Learning
2.7. Accommodating Learners
2.8. Ensuring No Duplication of Effort
2.9. Summary
3. The Research Methodology
3.1. Introduction
3.2. Selection of the Research Methodology
3.3. Ensuring Trustworthiness of the Study
3.4. Data Collection Methods
4. Intervention Design and Application
4.1. Introduction
4.2. The Intervention – Design & Development
4.3. Pilot Design
4.4. Intervention – Iteration One
4.5. Iteration Two
4.6. Intervention – Iteration 3
4.7. Intervention – Iteration 4
5. Data Collection & Results
5.1. Introduction
5.2. Data Collection Programme
5.3. Data Collection
6. Discussion, Conclusions & Recommendations
6.1. Introduction
6.2. Summary of the Study
6.3. Major Findings
6.4. Conclusions
6.5. Implications for Teaching & Action
6.6. New Design Methodology
6.7. Recommendations for Future Research
6.8. Concluding Remarks
7. References

Related Posts