CHAPTER 3 RESEARCH DESIGN AND METHODS
The research began with a literature review on the interrelationship between experiments, laboratory work and practical work. This chapter presents the research design, sample, data collection instruments and procedure followed when collecting data. The methods for data analyses are discussed. The research methods were based on the aim and the research questions as outlined in chapter 1 (see 1.5).
The population for this study is all the Foundation Physics students at Medunsa. Fifty two students were enrolled for Foundation physics. These students were divided into two groups during the practical work, one group (24) were doing electric circuits whereas (28) were doing magnetism. The available sample was 24 students. From the 24 students, 23 volunteered to take part in the research but 3 students did not come for all the tests. The other students lost interest along the way, hence the researcher ended up with 20 students. The twenty students were then randomly selected by arranging their surnames in alphabetical order and the first one was taken to the experimental group and the second one to the control group until the last one; at the end ten were in the experimental group and the other ten in the control group. This study focused on two groups, one group that received a treatment (the experimental group) and the other group that did not receive a treatment (the control group). Both the experimental and control groups wrote all the tests that will be discussed later. The experimental group did two experiments with simulations. After the intervention both groups went to the laboratory to do the laboratory work. Simulations were the treatment that the experimental group received. Because this study is based on electric circuits, the practical work was limited to two experiments on electric circuits. The experiments were “series and parallel connection of resistors, and Ohm’s law”. The duration for each experiment was three hours. Both experiments were also done in the laboratory with real apparatus.
A research design is guided by what the researcher wants to know and how data will be collected (Mark, 1996:28). According to Bless and Higson-Smith (1995:63), a research design is the planning of any scientific research from the first to the last step as for example shown in the flow chart (see figure 3.1). It is pertinent in a research study that the researcher specifies the major procedures he/she adopted. The study adopted the case study design. In the light of this research, a case study design was the most appropriate design to successfully investigate the problem and to serve the purpose of this study. The case is a unit of analysis: it can be an individual, a family, a work team, a resource or an institution. Each case has within it a set of inter-relationships which both bind it together and shape it, but also interact with the external world (Edwards & Talbot, 1999:51).
This case study describes the Foundation Physics students at the University of Limpopo who are doing practical work. A case study is an “empirical enquiry that explores a contemporary phenomenon within its real-life context when the boundaries between the phenomenon and context are not clearly evident and in which multiple sources of evidence is used” (Yin, 1994:13). Creswell (2003:15) define case study as “researcher explores in depth a program, an event, an activity, a process, or one or more individuals”. A case study should have a defined time frame; therefore this study was done for a period of six months.
Data collection instruments and procedure
Data was collected from the two groups. The students were initially given the DIRECT test to evaluate their prior knowledge of electric circuits as well as their understanding (see 3.4.1). This test was used as baseline information to evaluate their understanding before the electricity module was introduced in class. On the test the space was provided where students had to indicate their reason for selecting their answer. Those reasons were analyzed. The second test was TISP (see 3.4.2), to test their integrated science process skills namely; identifying and controlling variables, stating hypotheses, operational definitions, graphing and interpreting data and experimental design. This test does not concentrate on physics only; it concentrates on science in general. The intention was to test whether the students can for example identify the variable given a statement before they do the practical work. Prior to the laboratory sessions, the experimental group went to the computer laboratory to do simulations and completed the worksheets (see appendix C).
Afterwards, both groups did the experiments in the laboratory and completed the same worksheets. While doing the experiments in the laboratory the researcher was observing the two groups by using an observation schedule (see table 4.2). DIRECT and TISP were administered as post tests by both groups after attending theory classes and completing the practical work. Again their reasoning on the DIRECT test was evaluated. In addition a class test (see appendix D) which consisted of conceptual questions was written in class. Finally, six students were identified for the interviews, to supplement the data collected. The students were selected for the interviews based on their performance on the DIRECT pre test. The procedure followed is shown in figure 3.1 and the instruments used are discussed next.
The Determining and Interpreting Resistive Electric Circuit Concepts Test (DIRECT) was developed to “evaluate students’ understanding of a variety of direct current resistive electric circuits concepts” (Appendix A). The DIRECT test has been designed for use with high school and college/university students (Engelhardt & Beichner, 2004). The test consists of 29 multiple choice questions but in this study the researcher gave the students an option to explain their choices in order to see that students do not just guess the correct answer but can motivate their option as well as evaluate their level of understanding. In the beginning of the semester students from the control and the experimental groups wrote the DIRECT to test their prior knowledge and understanding on electric circuits. At the end, after the intervention, the students wrote the same DIRECT as a post test as indicated in figure 3.1.
In this study TISP developed by Kazeni (2005) was used and the reason for using TISP is to test students’ integrated science process skills and because it is developed in South Africa it does not use foreign examples and technical terms (Appendix B). From the theoretical model (see figure 2.3) skills development is part of quadrant 3. The students are expected to develop some skills in the laboratory when performing the experiments and then write the TISP post test. The integrated science process skills that TISP tests are:
• Stating Hypotheses – stating the proposed solutions or expected outcomes for experiments. For example, the students were expected to state what will happen to the current when the resistors are connected in series and in parallel. • Identifying and Controlling Variables – stating the changeable factors that can affect an experiment. It is important to change only the variable being tested and keep the rest constant. The one being manipulated is the independent variable; the one being measured to determine its response is the dependent variable; and all variables that do not change and may be potential independent variables are constants. For example in the Ohm’s law experiment, the resistance of a resistor was kept constant and the potential difference across the resistor was varied and the value of the current was recorded.
• Experimental Design – carrying out an experiment by carefully following directions of the procedure so the results can be verified by repeating the procedure several times. • Graphing and Interpreting Data – interpreting data statistically, identifying human mistakes and experimental errors, evaluating the hypothesis, formulating conclusions, and recommending further testing where necessary. For example, the students were required to draw three graphs on the same axes and determine the value of the resistor from the graphs. They had to draw a conclusion from those graphs. • Operational Definitions – explaining how to measure a variable in an experiment.
The students have to take a module on Electricity and Magnetism during the second semester of the academic year. The researcher developed the worksheets (see appendix C) for the practical work of this module. The same worksheets were used for both simulations and laboratory work with the intention of looking at the contribution of simulations on the experimental group. Indicated in quadrants 2, 3, and 4 of the theoretical framework model (figure 2.3), the laboratory work was purposefully done to address the components, lab experiment, skill development, demonstration and directed activity. The questions on the worksheets were testing students’ conceptual understanding rather than calculations. This was another change that was introduced.
During the practical work in the laboratory where students were performing the experiments, the researcher made some observation to check whether there was a difference between the two groups in terms of the development of science process skills. The basic process skills were also checked but this study was focusing mainly on the five that were mentioned earlier (see 3.4.2). An observation schedule was drafted to include both experimental and control group. These two groups were compared at the end.
The students did electricity and magnetism as part of their theory module. The researcher was not teaching the theory, but asked the lecturer concerned to allow her to suggest test questions when the students wrote their class test on electricity. The class test was written to see whether the students can transfer what they did in the laboratory to class and whether they can relate theory and practical work. The class test falls in the reception/meaningful quadrant of the theoretical framework. The role of the class test was to check whether the students can give a creative feedback after the directed activity. The experimental group was compared with the control group on the class test.
Interviews were selected as one of the research tools in this study as indicated in figure 3.1 to verify the data. Follow up questions were also taken into consideration when interviews were conducted since “interviews can encourage respondents to develop their own ideas, feelings, insights, expectations or attitudes, and it allows respondents to say what they think” (Richardson, nd).
During the interviews the researcher was taking notes. Trustworthiness of the interview can be interpreted as the cooperativeness, consistency, and confidence of the participants/ interviewees in responding to the questions asked. The interviews consisted of open ended questions in order to elicit richer, more complex responses and they are perceived as less threatening (Richardson, nd). The interviews were conducted in the laboratory where they were doing their experiment because the researcher thought that the environment was non threatening, and the students were encouraged to respond as freely as possible and they were assured that their responses would not affect their academic results because their marks were already submitted to the head of the department.
Reliability and validity of the instruments
Reliability and validity are crucial criteria in assessing the quality of a research study (Seale, 2004). All the instruments used in a study should be tested for their reliability and their validity as Engelhardt & Beichner (2004:102) stated “for the test to be useful and its results to be accurately applied and interpreted, it must be reliable and valid”. Even though the reliability and the validity of the instruments used in this study were tested by their developers, it was taken into consideration that the tests were used in another context, and then reliability and validity were checked. The tests were given to the two physics lecturers and a chemistry lecturer to check the content. The tests were also given to the physics I group to check whether they are familiar with the language used and whether they can finish writing within the time frame given. Reliability relates to the “extent to which an instrument provides similar results every time it is administered to the same sample at different times” (Engelhardt & Beichner, 2004:102). Validity is the “extent to which a test measures what it claims to measure and is not a quality that can be established in a single measurement, but accumulated via several measurements” (Engelhardt & Beichner, 2004:103).
The test used in this study was administered twice to the same groups at different times, so the testretest method as well as the internal consistency method was established by the Kuder-Richardson formula 20 (KR-20). The developers of DIRECT 1.0 found the KR-20 to be above 0.70 which is the acceptable value for the test to be reliable (Engelhardt & Beichner, 2004:102).
Validity of an instrument may be shown through content validity and construct validity (McMillan & Schumacher, 2001). During the development of the test, content validity was established by presenting the test and objectives to an independent panel of experts to ensure that the domain was adequately covered (Engelhardt & Beichner, 2004:103). The construct validity of DIRECT was evaluated through factor analysis and interviews; the factor analysis analyzes the interrelationships within the data and can be used to select groups of items that seem to measure the same idea (Engelhardt & Beichner, 2004:103).
The validation of DIRECT was done as follows: firstly experts were given the test to check the content; secondly a detailed factor analysis was performed to create and verify existing categories of questions; lastly the students were interviewed to confirm the clarity and meaning of questions (Engelhardt & Beichner, 2004:104).
Kazeni (2005) determined the reliability of this test instrument in two ways: firstly, the internal consistency reliability of the test by using the split half method secondly, the alternative form reliability by correlating scores from TISP and TIPS using the Pearson product–moment coefficient.
According to the test developer (Kazeni, 2005) the instrument was tested for content validity by six peer evaluators (raters) who comprised two Biology lecturers, two Physics lecturers, and two Chemistry lecturers from the University of Limpopo. The raters were given the test items and a list of objectives, to check the construct validity of the test by matching the items with the corresponding objectives. The test is intended for students, the lectures are only checking the validity. Other factors such as its discrimination power, the difficulty level, its reliability, and the different forms of bias may affect the validity of a test and those factors were determined during the development of the test (Kazeni, 2005).
The worksheets were compiled by the researcher looking at the worksheets compiled at Colorado by the PhET group. The questions selected for the worksheets were to test their understanding of the basic electric circuits’ concepts. If the students understand the basic concepts, it could be easy to understand the rest of the module. After the worksheet questions were finalised, it was taken to the theory lecturer and the head of the department who was lecturing Physics I group to check the content. It was then given to students from the Physics I group to complete in order to see whether they can finish in time and understand what is required in terms of the language used. This worksheet was given as their pre experiment and they did not know that it was meant for the Foundation students.
3.5.4 Class test
The class test was developed by the researcher after observing the students in the laboratory. The researcher noticed that most students have a problem of interpreting the diagrams and graphs. An inability to interpret equations, diagrams and graphs underlies many conceptual and reasoning difficulties (McDermott, 1993). The researcher chose the questions in the class test to check whether the students understood what they did when they were doing practical work and whether they can be able to relate it with what they did in class. The questions had diagrams to see whether the students can interpret them and give the correct explanations of those diagrams. Research has shown that traditional instruction does not challenge but tends to reinforce a perception of physics as a collection of facts and formulas. Students often do not recognize the critical role of reasoning in physics, nor do they understand what constitutes an explanation. They need practice in solving qualitative problems and in explaining their reasoning (McDermott, 1993). That was the reason why the class test questions were more on explanation than on calculations. The class test was also given to their theory lecturer and the head of the department to check the content as done with the worksheets.
As the aim of the study is to look at the contribution of simulations to the practical work of Foundation Physics students at the University of Limpopo, the research question was answered by data collected by all the instruments used in the study namely: DIRECT and TISP pre and post tests, worksheets, observations, class test, and interviews. The data was analyzed qualitatively. The quantitative data was used as a supplement to the data analysed qualitatively. The interviews were used to verify the data collected by other instruments.
Ethical principle “requires that participants should be kept informed about the aim of the research and researchers should protect the participants from harm” (Gay & Airasian, 2003). For this study the students who agreed to take part were informed about the purpose of the study in class. The researcher also attached the cover letter on the test instrument which states the purpose of the research. Students were assured confidentiality and that the interviews were only meant for research purposes. All the marks were submitted before the interviews were conducted.
Summary of the chapter
The study was conducted at Medunsa in the Physics Department. The study is a case study and the researcher chose to give a treatment to a certain group of students in the class. Two instruments, TISP and DIRECT were used to collect data before and after the intervention. The DIRECT test comprised two parts namely, multiple choice and reasoning part. The latter part was used to analyze the data qualitatively. The data from the worksheets for simulations and practical work as well as the class test were also analyzed qualitatively. Observations were made while the students were doing the experiments in the laboratory. All the instruments used for data collection supplemented each other so that four types of practical work (directed activity, demonstration, skill development and laboratory experiment) from the theoretical framework model were fulfilled. The students were interviewed to verify the data collected. The flow chart gives an indication of the procedure followed when collecting data. The fact that the experimental group had more exposure of the content done “by means of simulations” than the control group, they tend to do better because they were exposed to the same worksheets twice. To avoid the design flaw in future studies, all the groups should be exposed to the same treatment if a study is a case study, unless if it is a comparison study and the researcher wants to see if the treatment can work or not.
TABLE OF CONTENTS
LIST OF FIGURES
TABLE OF CONTENTS
CHAPTER 1 1 ORIENTATION TO THE STUDY
1.2 Rationale of the study
1.3 Context of the study
1.4 Aim of the Research
1.5 Research problem and research question
1.6 Definitions of terms
1.7 Summary of the chapter
CHAPTER 2 LITERATURE REVIEW
2.2 History of the laboratory in science
2.3 Role of practical work
2.4 Laboratory environment
2.5 Problems with practical work
2.6 Practical manuals
2.8 Instruments used to measure electric circuits concepts
2.9 Process skills
2.10 Students’ understanding
2.11 Foundation programme
2.12 Theoretical framework
2.13 Summary of the chapter
CHAPTER 3 RESEARCH DESIGN AND METHODS
3.2 Research sample
3.3 Research design
3.4 Data collection instruments and procedure
3.5 Reliability and validity of the instruments
3.6 Data Analysis
3.7 Ethical Issues
3.8 Summary of the chapter
CHAPTER 4 RESULTS AND DATA ANALYSIS
4.5 DIRECT Post test
4.6 TISP Post test
4.7 Class test
4.8 Analysis of questions from Class test
4.9 Comparing the six students
4.10 Interviews and their analysis
4.11 Summary of the interviews
4.12 Summary of the chapter
CHAPTER 5 SYNTHESIS, RECOMMENDATIONS AND CONCLUSIONS
5.2 Summary of the study
5.3 Discussions of findings
5.4 Limitations of the study
5.5 Recommendations Suggestions for future research
GET THE COMPLETE PROJECT
THE CONTRIBUTION OF SIMULATIONS TO THE PRACTICAL WORK OF FOUNDATION PHYSICS STUDENTS AT THE UNIVERSITY OF LIMPOPO