Motivation and Self-identity in Mainland China

Get Complete Project Material File(s) Now! »

Chapter 3 Methodology

Introduction

To address the research questions posed at the end of last chapter, this doctoral study mixes methods by utilising semi-structured interviews and a statistically analysed self-report survey. Specifically, Study 1 employed a retrospective case study design with 20 English language learners from China enrolled in a PhD programme at the University of Auckland. The goals of the study were to (1) describe learners’ motivations for and self-identities in learning English, (2) identify changes in motivations and self-identities across the trajectory of learners’ life and study experiences, and (3) identify possible factors contributing to their success in English learning. Study 2, using ten Chinese students enrolled at the University of Auckland, trialled and evaluated potential survey instruments and selected appropriate items and scales for use with the target population. Study 3 employed a cross-sectional online questionnaire survey of 443 postgraduate and undergraduate Chinese students undertaking tertiary study in New Zealand. The goal was to explore structural relations among second language (L2) regulatory styles, identity changes and L2 motivational possible selves so as to describe the development of L2 learning. This chapter consists of five parts: (1) the logic of mixing methods, (2) sampling, (3) semi-structured interview study, (4) quantitative survey study, and (5) research ethics.

Mixing Methods

Mixing methods research is an approach that combines the elements of quantitative and qualitative approaches so as to provide a broader and deeper understanding of a research problem or a question (Johnson, Onwuegbuzie, & Turner, 2007). While strictly this approach mixes methods, it is commonly called ‘mixed methods’ which implies that some sort of a new hybrid method has been created. In this thesis, each method has been used according to conventions and standards appropriate to the method. The mixing takes place through the use of results and insights from each study to inform the design of the subsequent studies and to inform a synoptic synthesis of results across all data sources.
According to Creswell (2014), mixed methods research is characterised by (1) collecting both quantitative and qualitative data and analysing the two forms of data rigorously, (2) integrating the two forms of data by merging (e.g., convergent parallel mixed methods design) or connecting (e.g., explanatory/exploratory sequential mixed methods), and (3) considering the timing of the data collection (i.e., concurrent or sequential) and the emphasis (e.g., equal or unequal) of the two forms of data.
As shown in the Figure 5 (Creswell, 2014, p. 220), in a convergent parallel mixed methods design, researchers firstly collect quantitative and qualitative data and analyse them separately, then compare the results to see if the two sources of findings confirm or disconfirm each other. In an explanatory sequential mixed methods design, researchers firstly collect the quantitative data and then analyse the data and use the results of quantitative data to plan the qualitative study. That is, the quantitative results decide the types of participants that will be selected for the qualitative study and the types of questions that will be asked of the participants (Creswell, 2014). The aim of this design is to use the qualitative findings to provide a more detailed explanation of the quantitative findings. By contrast, in the exploratory sequential mixed methods design, researchers collect the qualitative data and analyse it and then use the results of qualitative study to inform the quantitative study. That is, the qualitative results are helpful in developing the new quantitative measures (Creswell, 2014). The aim of this design is to see if the data from a small sample (qualitative study) can be generalised to a large sample of a population (quantitative study).
As Creswell states (2014), the rationale for choosing mixed methods should be: it combines both quantitative and qualitative research and minimises the limitations of both approaches. In other words, the mixed methods design can either confirm or disconfirm the data from different approaches (quantitative versus qualitative), or it can complement the weaknesses of each approach. Specifically, the two main research designs of the quantitative approach are surveys and experiments. For survey design, the strength of quantification is to generalise or draw inferences to the population; and for experiment, its strength is to test the influence of a treatment on an outcome (Creswell, 2014). However, if the quantitative study is highly controlled (e.g., an experimental study), it is hard to confirm its validity—that is, how close the research situation is to real life (Carr, 1994). In addition, quantitative study is inflexible, whereas qualitative study is exploratory in nature and is flexible and dynamic (Duffy, 1987). The strength of the qualitative approach is its particularity, since it provides detailed description and themes developed in a specific site (Creswell, 2014). Compared with the quantitative approach, the limitation of the qualitative approach is a lack of robust generalisation.
Previous research into motivations and selves of L2 learners has not used an exploratory sequential mixed methods design. Furthermore, previous inquiries into whether the two possible selves of the L2 motivational self system (Dörnyei, 2005, 2009) are sufficient for Chinese learners of English, who reside in an exam-driven education system (Yu & Suen, 2005) and a collectivistic culture, have not used a qualitative approach. Hence, this thesis employs an exploratory sequential mixed methods design in which qualitative data and analysis are conducted prior to a quantitative phase. That is, qualitative data inform the design of a quantitative survey and the quantitative data build on the results of the qualitative data.
The aim of this research design is to develop some measurements from the qualitative data and then to see if the interview findings can be generalised to a large sample in the quantitative phase (Creswell, 2014), and also to test the structure and relations among these constructs. In effect, this exploratory sequential mixed methods design (see Table 1) has three phases, which align with the three studies described earlier. The first phase is exploratory, based on semi-structured interviews, and the second phase is the qualitative judgement of instrument validity. The third is a quantitative large-scale survey administered in order to explore and establish self-reported attitudes of a population. Specifically, factor analytic techniques were used to handle the hypothesised latent factor structure of the survey items, followed by causal correlational analysis to explore the structural relations among L2 regulatory styles, identity changes and L2 motivational possible selves.

Sampling

The quality of research depends on choosing an appropriate methodology and suitable sampling strategy (Morrison, 1993). Understandably, it is very hard for researchers to gain access to the whole population of interest. Hence, samples from the whole population of interest are used. The information obtained from a sample is expected to be representative of the whole population being studied (Cohen, Manion, & Morrison, 2005). Further, four important factors in sampling need attention; that is, sample size, representativeness of the sample, access to the sample and sampling method.

Sample Size

The decision on sample size relies on research aim and the nature of the population being studied as well as the methodology approach (e.g., qualitative, quantitative) that researchers have chosen (Cohen et al., 2005). A quantitative approach (e.g., survey) usually needs a large sample, perhaps each variable with six to ten cases, whereas a qualitative approach (e.g., case study) tends to require a small sample. A larger sample provides greater reliability and allows more sophisticated and advanced statistics. Thus, the larger the sample the better. Additionally, the sample should be large if (1) there are many variables, (2) variables are heterogeneous, (3) the relationship between variables is expected to be small, (4) reliable measures of the dependent variables are unavailable, and (5) there are many subgroups in the sample (Borg & Gall, 1996).
Further, for quantitative data, margin error (i.e., the amount of error that researchers can tolerate), confidence level and the total number of the whole population should be taken into consideration in determining a required sample size. Generally, a 5% margin of error in the probability sampling and a 95% confidence level are recommended (Raosoft, 2013). For example, using probability sampling, researchers should recruit at least 377 participants in a target population of 20,000, in order that if the survey of 377 participants were to be repeated many times, 95% of the time the survey response would lie within ±5% variation range of the true response of the whole population (example from Raosoft, 2013). However, the required size will not increase steeply when the target population is more than 20,000. That is, the required sample size for a target population of 1,000,000 is only 384 participants (Cohen et al., 2005, p. 104).

Sample Representativeness

Researchers need to consider whether a sample is valid; that is, whether it can be representative of the whole population being studied. Researchers should keep in mind the representativeness of this sample and how to design the sampling frame correctly and clearly (Cohen et al., 2005). Some variables (e.g., climate) should be taken into account if they can exert influence on the research.

Sample Access

Access is a factor that should be considered in the early stage of a study. Researchers should be aware that access to the target population is likely to be permitted, perhaps by getting the approval of an institution’s ethics committee, and practicable, with the potential for the participants’ participation (Cohen et al., 2005). Researchers should consider how access to the potential participants will be undertaken; that is, consider who it is they may have to contact in order to get access to participants. Whether the study results will be permitted to be published is another concern for researchers prior to carrying out a study. For the current study, these concerns were addressed while preparing and submitting the Ethics application to the University of Auckland Human Participants Ethics Committee (UAHPEC).

Sample Method

There are two kinds of sampling method: probability sampling and non-probability sampling. In the former, every member of the target population has an equal chance of being included in the sample. In the latter sampling method, for every member the chance of being included in the sample is not equal—that is, some are included but others are excluded (Cohen et al., 2005). The latter sampling method is targeting a particular group. In this doctoral study, non-probability sampling—convenience sampling and snowball sampling—was used in both a semi-structured interview study and a survey study.

Convenience sampling.

Convenience sampling is sometimes referred to as accidental sampling (Burnard, 2004; MacNealy, 1999), opportunistic sampling (Barton, 2001) or haphazard sampling (Kalton, 1983). Different researchers offer different definitions of convenience sampling. According to MacNealy (1999), convenience sampling means that the researchers visit public locations and invite the people they meet to participate. Yet, Higginbottom (2004) emphasises the ready availability characteristic of this sampling technique (e.g., potential participants are easy access or contact). However, Koerber and McMichael (2008) comment that although this sampling is readily available, researchers still need to put some effort into recruiting the potential participants for convenience sampling.
Convenience sampling is considered to be a study limitation in many disciplines (Barton, 2001), since the participants that researchers recruit are likely to be similar to each other. Further, this sampling method might lead to bias in the data, as the participants might have some similarity if they come from the same educational institution or clinic, and volunteer participants would be more skilled and competitive (Lunsford & Lunsford, 1995). Another disadvantage of this sampling method is that researchers do not know what wider population this sample can represent or how this sample is different from other potential samples (Tansey, 2007). Thus, there might be a limitation on its robust generalisation. It might be risky to use this sampling data to make inferences to the general population (Kalton, 1983).
Nevertheless, convenience sampling method can save time, energy and money (Marshall, 1996). Currently, it seems to be the most commonly used sampling method in clinical research because it is easy and fast yet the least expensive and troublesome (Lunsford & Lunsford, 1995). It can generate rich data, because the close relationship between researchers and the researchers’ site is helpful in getting access to potential participants (Koerber & McMichael, 2008). In this way, the method ensures the richness of the data, which might not be reached if the sample were not familiar and thus not convenient to the researcher.
Overall, the availability of convenience sampling is a ‘double-edged sword’. It can guarantee the richness of the data, but it cannot be overgeneralised due to a certain amount of similarity between potential participants.

Snowball sampling.

Traditionally, snowball sampling has been used to inquire into one’s immediate social environment by asking social relations questions of the interviewee (e.g., who is your best friend?) for sampling purposes. Thus, this sampling method is involved in two populations: the individual population and the individual’s social relation population (Coleman, 1958). Further, Coleman argues that this sampling method was uniquely designed for sociology research as it took into account interpersonal relations. In effect, snowball sampling can be used in sociology for identifying the number of social relationships as well as in obtaining the non-probability sample in other research fields (Thompson, 2002). It can be used in easy-to-reach populations and hard-to-reach populations (Goodman, 2011). Also, many researchers believe that snowball sampling can be used in various research methods and designs and it is well suited for many research purposes (Marshall, 1998; Patrick, Pruchno, & Rose, 1998), particularly for the study of sensitive issues (e.g., opiate addiction study; Biernacki & Waldorf, 1981).
Snowball sampling selects the sample through a social network and, therefore, the disadvantage is that the members of the sample may all belong to a particular group or have a certain bias (Kumar, 2005). That is, the choice of the overall sample relies on the first group of participants who are helping researchers to identify other participants. Further, the problems of this chain referral sampling lie in the following aspects (Biernacki &Waldorf, 1981): (1) initiation of the chain referral and make sure that this chain will continue, (2) verification of potential participants’ eligibility, and (3) control of the chains and of the number of cases in each chain.
However, snowball sampling can reduce potential participants’ scepticism and thus reduce their reluctance to participate, because of its referral chain (Streeton, Cooke, & Campbell,2004). In other words, this sampling method seems to increase response rates, as people are more likely to accept a participation invitation if they know the one extending the invitation or if they are referred by their acquaintances. In addition, the cost of snowball sampling is cheaper than other sampling methods (Patrick et al., 1998).

Sampling Concerns

In the semi-structured interview study, sixteen participants were recruited through convenience sampling. Four participants were then recruited via snowball sampling by those recruited into the study through convenience sampling. In order to avoid the disadvantage of snowball sampling method, as previously stated, particularly the possibility of data bias resulting from members of one particular group or organisation, the initial four students were chosen across different faculties at the University of Auckland (i.e., Faculty of Medicine, Faculty of Business, Faculty of Arts, and Faculty of Engineering). In relation to eligibility verification, participants were asked to show their student ID card and their doctoral registration letter to confirm their Chinese doctoral student status.
In order to avoid the disadvantage of convenience sampling, particularly the weakness of generalisation possibly resulting from the sample coming from the same institution, a large-scale survey study recruiting participants from different universities in New Zealand was conducted, in the hope of compensating for the weakness in this convenience sampling and semi-structured interviews study. In the survey study, 443 Chinese university students from different universities in New Zealand were recruited, which accounted for 3.7% of approximately 12000 Chinese university students in New Zealand universities in 2013, according to the report of New Zealand universities trends in international students (Ministry of Education New Zealand, 2013). This would create a margin of error of 4.57 % for a random sample (Raosoft, 2013); however, the current sample arose from non-probability sampling (i.e., snowball and convenience sampling) and, therefore, the margins of error relative to the population are less robust. Yet, this sampling size is large enough to undertake the sophisticated statistical analysis which normally would require the sample of 400 or more.

Semi-structured Interviews Study

The nature of this semi-structured interview study is exploratory, in the hopes of offering “a window-like” view on the specific situation that the researcher is studying (Giacomini & Cook, 2000, p. 480). In this doctoral thesis, the semi-structured interview study used both thematic analysis and frequency in data analysis.
Semi-structured interviews incorporate a set of questions used in structured interviews as well as reflecting the open-ended and explorative nature of unstructured interviews (Wilson, 2014). Semi-structured interviews use questions, prompts and other resources to draw interviewees fully into the research topic; they combine open-ended questions with theory-driven questions in order to elicit the data which are grounded in the interviewees’ life/study experience as well as being guided by the existing frameworks of the researchers’ research field (Galletta & Cross, 2014). Semi-structured interviews focus on a set of predetermined open-ended questions, and new questions emerge from the conversation between interviewer and interviewee (Dicicco-bloom & Crabtree, 2006). Thus, semi-structured interviews reflect more flexibility and are useful in probing interviewees’ attitudes.
Although it takes a longer time to conduct semi-structured interviews and it is harder to carry out data analysis, semi-structured interviews help the interviewer to develop a rapport with respondents and thus to probe a new area so as to generate more data (Smith, Harre, & Langenhove, 1995). Semi-structured interviews look like free conversations (Fylan, 2005), rather than asking a series of questions, and the conversation can be easily changed between participants. This flexibility of semi-structured interviews makes it more suitable for addressing a ‘why’ question than a ‘how many’ question. Additionally, semi-structured interviews create a more stress-free atmosphere and thus they are appropriate for discussing a sensitive or private topic. It is worth noting that the interviewer should steer the interview subtly, instead of asking a leading question.

Table of Contents
Abstract
Acknowledgements 
Co-Authorship Form 
List of Figures 
List of Tables
Chapter 1 Introduction 
1.1 Problems
1.2 Guiding Principles
1.3 Thesis Organisation
Chapter 2 Literature Review
2.1 Introduction
2.2 Motivation
2.3 Identity
2.4 Motivation and Self-identity in Mainland China
2.5 Research Questions
Chapter 3 Methodology
3.1 Introduction
3.2 Mixing Methods
3.3 Sampling
3.4 Semi-structured Interviews Study
3.5 Quantitative Survey
3.6 Research Ethics
Chapter 4 Semi-structured Interviews Study 
4.1 Introduction
4.2 Method
4.3 Results
4.4 Summary
Chapter 5 Survey Study 
5.1 Introduction
5.2 Method
5.3 Results
5.4 Summary
Chapter 6 Discussion and Conclusion 
6.1 Introduction
6.2 Summary of Main Findings
6.3 Discussions of the Main Findings
6.4 Limitations and Future Studies
6.5 Implications
6.6 Significance of the Study
References
GET THE COMPLETE PROJECT
L2 Motivations and Self-identities of Chinese Learners of English