Feedback on discourse in individual student scripts

Get Complete Project Material File(s) Now! »

CHAPTER THREE: METHODOLOGY

The purpose of this chapter is to give a detailed account of the research design, to outline principles behind the choice of research methodology, and to provide an overview of the data collection and data analysis in the current thesis study. It begins with the research questions in the first section and then specifies the research tradition for this study in the second. The third section explains why the case study approach is particularly appropriate for this study. The fourth section reports on the pilot study, the main purpose of which was to inform my research design and help establish the validity and reliability of the data collection instruments employed in the main study. Based on the results of the pilot study, it reports on the changes that were considered necessary before carrying out the main study. It ends with a detailed description of the settings and participants, the process of data collection, the sources of data and the instruments and data analysis of the main study. The eighth and ninth sections explain the strategies used to enhance the quality of the study and its ethical integrity.

Research questions

Conducted in China, this case study aimed to achieve a deep understanding of three experienced in-service teachers’ feedback at the discourse level in EFL writing classes for tertiary students. The purpose of this study was to analyse and understand in detail the phenomenon of teachers’ discourse-focused feedback in terms of the features that teachers provide feedback on, the strategies they use, and the reasons why they provide the feedback that they do. This study was also interested to establish a more general picture of EFL teachers’ discourse-focused feedback at tertiary level. Specifically this work addresses the following questions:
RQ1: To what extent do the teachers provide feedback on discourse features in learners’ writing?
RQ2: What strategies do the teachers use to provide feedback on discourse?
RQ3: What are the teachers’ beliefs about providing feedback on discourse? How do teachers’ feedback practices relate to their beliefs?
Below are the definitions of some of the key terms used in this study (more details can be found in Chapter 2):
•leraners ’ writing: student scripts completed after class by tertiary level English language learners.
•discourse features: properties dealing with the writers’ use of language in writing to
relate to their readers, which can help to establish a particular genre (e.g., the argumentative or narrative essay) or construct text in general (e.g., organisation, unity, coherence, cohesion, and metadiscourse) to construe meaning as a whole.
•feed back practices: primarily refer to teachers’ written feedback about individual students’ scripts, specifically what they focused on and the strategies they used. The verbal
feedback to the whole class that followed written feedback is also included as supplementary evidence to assist with the understanding of teachers’ written comments.
•feedback strategy: the ways that teachers provided feedback on the students’ scripts in
terms of overall orientation (praise or advice), the directness of advice (correction, suggestion, or identification of problems) and the location (running or end comments).
•teachers’ beliefs: “Pychologically held understandings, premises and propositions about the world that are felt to be true” (Richardson, 1996:103).

Research tradition

A research tradition represents a patterned set of assumptions concerning reality, knowledge of that reality and the particular ways of knowing about it (Guba, 1990). Research is designed and conducted with the guidance of “a set of beliefs and feelings about the world and how it should be understood and studied” (Denzin & Lincoln, 2003:33). This study sits within the qualitative-interpretive research paradigm and in this section, I give a brief overview of the main features of this tradition.

Interpretive epistemology

Epistemology is concerned with the nature and scope of knowledge, and the reliability of claims about the status of knowledge (Schwandt, 2003). An interpretive approach is taken when researchers conduct research with the view that social phenomena are different from physical phenomena, that all human action is inherently meaningful, that knowledge is an individual and social construct of the human mind, and that individuals’ perceptions of the world can, therefore, be understood and interpreted in different ways (Hammersley, 1992; Schwandt, 2003). As Fenstermacher says, “Pristine, absolute, unfettered truth is neither possible nor required in the social sciences” (1994:45). Therefore, “any hope of discovering laws of human behaviour is misplaced…since human behaviour is continually constructed,and reconstructed, on the basis of people’s interpretations of the situations they are in” (Hammersley & Atkinson, 1995:8).
Interpretive studies attempt to understand phenomena through the meanings that people assign to them. This study sits within the interpretive tradition as it was concerned with interpreting teachers’ beliefs and practices in order to understand what teachers do and why they do them. Interpretive methods of research aim to “produce an understanding of the context of the information system, and the process whereby the information system influences and is influenced by the context” (Walsham, 1993).

Qualitative methodology

If the interpretive position is that there is no objective reality in the social world, and that individuals interpret the world differently, then research methodology needs to reflect this position. Qualitative research is concerned with “studying things in their natural settings, attempting to make sense of, or interpret phenomena in terms of the meanings people bring to them” (Denzin & Lincoln, 1994:2), and with “trying to develop some insights we can work with” (Eisner, 2001:138). It aims to challenge assumptions and provide insights into human behaviour, enabling the researcher to “explore the complexities and conundrums of the immensely complicated social world we inhabit” (Richards, 2003:8). Qualitative researchers try to get closer to an individual point of view through detailed observation and interview, in order to attain “thick descriptions” (Geertz, 1973).
In this study, qualitative data was used to develop thick descriptions of each teacher’s feedback practices and beliefs. Some quantitative data was used to help build up a picture of the complexity of each teacher’s practices. I will now explain the case study approach that I used for this study in detail.

Case study approach

As I have mentioned in chapter 2, there is a lack of research about teachers’ feedback on discourse features. If very little is known about a phenomenon, the detailed study of a few cases is particularly appropriate because it does not rely on previous literature or prior empirical findings (Eisenhardt, 1989). I now outline why the case study approach was chosen, and summarise the main features of this approach with a particular focus on why it was suitable for this study.

Features of a case study approach

The case study is an in-depth exploration from multiple perspectives of the complexity and uniqueness of particular phenomena in their real-life context (Dornyei, 2007; Duff, 2008; Stake 1995; Yin 2003). The main features of a case study approach in relation to this particular research are as follows:
Uniqueness and particularity: case studies are often conducted in contexts that are unique rather than generalisable (Yin, 2003). They explore the particularity of the case (Stake, 1995) and emphasise the detailed analysis of a small number of events or conditions to reach understanding of their relationships in their natural contexts (Cohen et al., 2000; Gall et al., 2007; Yin, 2003). This study of teachers’ feedback practices and beliefs was classroom-based research in a typical higher education teaching environment of in China where EFL writing is taught to tertiary students within a genre-based curriculum.
Complexity and thick description: case studies provide multiple perspectives (Yin, 2003) to explore the phenomena by using multiple data collection to get “thick descriptions” (Geertz, 1973) of the case. This study aimed to describe teachers’ feedback on discourse features holistically and explore in depth why they provided feedback in the way they did. Only by involving a small number of individuals (three cases), is it possible to understand the complexity of a case in the most complete way possible.
Multiple sources of data: in order to conduct a rich description of the case, this research involved the use of multiple methods for collecting data, including observation, document analysis, and interviews with stimulated recall protocols. Rich understandings of a case were attained by using multiple sources of data. This data included document data (students’ scripts with teachers’ written feedback on them), transcriptions of verbal feedback observation (verbal feedback to the whole class after providing written feedback), and interview transcriptions (the initial interview held before the first observation and post-observation interviews with stimulated recall protocols, after both written and verbal feedback was given).
Flexibility and rigorous design: case study research is a dynamic process that evolves as activities unfold and elements of the research process interact, thus impacting on one another (Anderson & Arsenault 1998:27). This necessitates the understanding of an ‘emergent strategy.’ Such flexibility requires a principled approach to the research design and rigorous procedures to ensure the full complement of data collection (Richards, 2003; Yin, 2003). In this study, flexibility was demonstrated through the different kind of probing questions that were used according to the teachers’ different replies to the main interview questions. In post-observation interviews, different stimulated recall protocols were used for different teachers to enable them to explain what they did in their feedback practices.

Types of case study

A case study can be defined in terms of the process of carrying out an investigation, the unit of analysis, or the end product (Merriam, 1998). In this study, a rich description of three cases emerged through presenting an in-depth understanding of the phenomenon of teachers’ discourse-focused feedback. Furthermore, it explored how their beliefs relate to their feedback practices. I now outline specific characteristics of the type of case study approach used for this study.
Descriptive: a descriptive case study presents a complete description of a phenomenon within its context, in order to examine it more fully or to differentiate it from other phenomena (Yin, 2003). This study was partly descriptive as it aimed to fully describe the teachers’ discourse-focused feedback in terms of the features that teachers provided feedback on and how they gave that feedback. The strategies and focus of feedback were carefully defined to conceptualise the discourse features and the strategies applied by teachers that arose from the data based on theoretical discussions in the literature.
Exploratory: an exploratory case study aims to identify issues that can be studied in subsequent research (Yin, 2003). This study not only looked at what teachers did when giving feedback, but also identified factors that prevent teachers from giving feedback in line with their beliefs. This is an issue that can be studied further in subsequent research about the different contexts for providing feedback on writing for English learners at tertiary level. The descriptive element of the study helped to build up a complete picture of the case, and in this sense it was also partly an exploratory case study that endeavoured to explain feedback practice.
Instrumental: an ‘instrumental’ case study (Stake, 1995) explores the case to help generate an understanding of wider issues beyond the case itself. This case study was instrumental because it aimed to facilitate an understanding of what how teachers give feedback and the relationship between teachers’ beliefs and actions, rather than only studying the cases because of their intrinsically interesting nature.
Multiple cases: choosing more than one subject provided the opportunity to closely study the same phenomenon from different perspectives and experiences (Duff, 2008). Mackey and Gass (2005) suggest that findings from multiple case studies can be examined together to help researchers draw stronger conclusions. In this study, each of the three teachers was treated as an individual case, and the findings of each single case provided information contributing to the whole study. The similarities and differences between the three cases were explored separately through cross-case analysis.
Analytic generalisation: case studies typically make use of ‘analytic generalisation’ (Yin, 2003), where findings are generalisable to theoretical propositions rather than to wider populations (Gall et al., 2007; Duff, 2008). Other researchers may investigate the findings of this research to see whether they are valid in other contexts or settings (Bassey, 1999). This case study used thick description to help readers decide the generalisable nature of the findings (e.g. recommendations/guidelines for discourse-focused feedback) according to their contexts.

Piloting of instruments and categories of analysis

An important element of research design is the piloting of data collection instruments and data analysis categories. Therefore, I conducted a pilot study in Beijing, before finalising the research design of the main study. Appendix 1 includes a detailed account of the procedures and findings of the pilot study. In this next section, I briefly describe the main features of the pilot study and explain how this helped inform my data collection and analysis for the main study. I will then discuss the research design of my main study in section 3.5.
Aim and overview of the pilot study
The main purposes of the pilot study were to:
•Assess the extent to which the data collected would answer the research questions.
•Trial instruments for data collection.
•Trial data collection procedures.
•Develop a coding scheme to identify the features that discourse teachers focus on, and the strategies applied in discourse-focused feedback.
I worked with one participant who had the pseudonym Jane, to explore her feedback practices and beliefs on discourse. This was achieved by collecting data in typical writing class settings over a two month period from May-June 2009 using document analysis, observation, and post-observation interviews. Figure 1 shows the process informing the design of this pilot study.

CHAPTER ONE: INTRODUCTION
1.1 Aims of the study
1.2 Context
1.3 Research stance
1.4 Overview of the study
CHAPTER TWO: LITERATURE REVIEW
2.1 Discourse-view of writing as text .
2.2 Teacher feedback
2.3 Teacher Beliefs
2.4 Summary
CHAPTER THREE: METHODOLOGY .
3.1 Research questions
3.2 Research tradition
3.3 Case study approach
3.4 Piloting of instruments and categories of analysis.
3.5 Research design of the main study.
3.6 Summary
CHAPTER FOUR: CASE ONE MEI
4.1 Profile
4.2 Feedback on discourse in individual student scripts
4.3 Strategies used in feedback on discourse
4.4 Whole class feedback
4.5 Relationship between beliefs and practices
4.6 Summary of key issues
CHAPTER FIVE: CASE 2 LAN
5.1 Profile.
5.2 Lan’s feedback on discourse features in students’ script
5.3 Strategies used when giving discourse-focused feedback
5.4 Whole-class feedback
5.5 Relationship between beliefs and practices..
5.6 Summary of key issues
CHAPTER SIX: CASE THREE QIANG
6.1 Profile.
6.2 Feedback on discourse in individual student scripts
6.3 Strategies used in discourse-focused feedback
6.4 Whole class feedback
6.5 Relationship between beliefs and practices
6.6 Summary of key issues
CHAPTER SEVEN: CROSS-CASE COMPARISON
7.1 Profile of the three teachers
7.2 Comparison of feedback practices and beliefs between the three cases .
7.3 Summary
CHAPTER EIGHT: DISCUSSION
8.1 To what extent do teachers focus on discourse features in feedback comments?
8.2 What strategies do the teachers use to provide feedback on discourse features?
8.3 How do teachers’ feedback practices relate to their beliefs?
8.4 What can we learn from the three teachers?
8.5 Summary
CHAPTER NINE: CONCLUSIONS AND IMPLICATIONS
9.1 Summary of main findings
9.2 Concluding remarks
9.3 Theoretical, pedagogical and practical contributions..
9.4 Limitations of the study
9.5 Suggestions for future research
9.6 Summary
GET THE COMPLETE PROJECT
Teachers’ Feedback on Discourse Features in EFL Writing: Case Studies in the Chinese Context