SciELO - Scientific Electronic Library Online

 
vol.28 issue3 author indexsubject indexarticles search
Home Pagealphabetic serial listing  

South African Journal of Education

Print version ISSN 0256-0100

S. Afr. j. educ. vol.28 n.3 Pretoria Aug. 2008

 

CONTENTS

 

The "movement" of mixed methods research and the role of educators

 

 

John W. CreswellI; Amanda L. GarrettII

IJohn Creswell is Professor of Educational Psychology at the University of Nebraska-Lincoln, USA, and has taught courses in qualitative and mixed methods research for 30 years. He is the author of 11 books, Co-Director of the Office of Qualitative and Mixed Methods Research at Nebraska, the founding Co-Editor for the Sage journal, Journal of Mixed Methods Research, and a Senior Fulbright Scholar. E-mail: jcreswell1@unl.edu
IIAmanda Garrett is a doctoral student in Educational Psychology at the University of Nebraska-Lincoln, USA, and a researcher in the Office of Qualitative and Mixed Methods Research at the University of Nebraska

 

 


ABSTRACT

The landscape of research is continually evolving, enabling researchers to study increasingly complex phenomena. Educational researchers have propelled much of this forward progress and have developed novel methodologies to provide increasingly sound and complete evidence. Mixed methods research has emerged alongside quantitative and qualitative approaches as an important tool for researchers. In this article our overall aim is to better acquaint educational scholars with the mixed methods field by articulating the development of the mixed methods field and by citing current trends and issues. The role of educational researchers in the evolution of mixed methods research is high-lighted. The early and ongoing dialogue of mixed methods research is multi-disciplinary in nature with current writings across fields. The current debate over key aspects of mixed methods research is now in progress and is ripe for future contributions. Even the very nature of what constitutes mixed methods research is being discussed among scholars. Understanding and advancing the mixed methods field is an important goal for methodologists and researchers. With the increased interest and enthusiasm for mixed methods research, it is likely that the dialogue surrounding mixed methods approaches will thrive, continuing the movement of the field.

Keywords: educational research; mixed methods


 

 

Orientation to the mixed methods field

At this moment in the development of research approaches, the educational researcher needs a large toolkit of methods and designs to address complex, interdisciplinary research problems. This researcher may be part of a team of researchers with individuals bringing to the table different research skills and training most likely skills in both quantitative and qualitative research (Creswell & Plano Clark, 2007). These skills are needed to study the increasingly complex problems facing educators and social scientists. Addressing these problems requires amassing substantial evidence all types of evidence gained through measurement of precise questions, as well as more general assessment through open-ended questions. New tools are continually added to the researcher's toolkit, because our approaches for scholarly inqui- ry continually evolve. As evidence for this, even a couple of decades ago the research community was not familiar with some of the approaches to sophisticated statistical analysis such as structural equation modeling (Kline, 1998) or some of the more community-based approaches to qualitative research such as participatory action research (Kemmis & Wilkinson, 1998). In short, research approaches continually evolve and change in response to the complex, interconnected global communities and their needs in our world.

This large toolkit now includes skills in quantitative and qualitative research. Quantitative research has traditionally provided a measurement orientation in which data can be gathered from many individuals and trends assessed across large geographic regions (Creswell, 2008). On the other hand, qualitative research yields detailed information reported in the voices of participants and contextualized in the settings in which they provide experi- ences and the meanings of their experiences (Creswell, 2008). When researchers bring together both quantitative and qualitative research, the strengths of both approaches are combined, leading, it can be assumed, to a better understanding of research problems than either approach alone.

This core assumption is the basis for an approach to inquiry called "mixed methods research". It has been described as a "third movement" in the evolution of research methodology following quantitative and then quali- tative and, as a "movement" it is seen as the prevailing way most educators will be approaching research (Tashakkori & Teddlie, 2003). However, it is early in this "movement". Despite substantial developments in mixed methods, the field of study is plagued by early growing pains such as debates and disagreements about core issues such as its essential nature. Lingering behind many thoughtful discussions about mixed methods are key stances of authors, many of which are not articulated clearly, and certainly not apparent to the casual reader of the mixed methods literature. A need exists to better understand the emerging nature of mixed methods and to clarify the stances of authors. A need also exists to examine the literature closely to see whose voices are being heard or at least presented. Such an exploration in this article, it would seem, would help those decide if mixed methods is appropriate to use, assess whether it is a legitimate inquiry approach, and consider how to apply the approach to a particular study.

 

Purpose and concept clarification

The purpose in this article is to briefly discuss the developments and trends of mixed methods that educators should be cognizant of and that educators have helped shape. The term "movement" is emphasized to suggest that mixed methods is a growing trend in research methodology. As a working definition, mixed methods is an approach to inquiry in which the researcher links, in some way (e.g. merges, integrates, connects), both quantitative and qualitative data to provide a unified understanding of a research problem (adapted from Creswell & Plano Clark, 2007).

A sub-text flowing throughout the discussion will be the important role that individuals in the field of education have played in the growth of mixed methods research. Our focus on the role of educators is not to minimize the contributions of others (and certainly others are mentioned); our focus is only to create a dialogue about the educators involved in mixed methods during its 20-year history. Discussing the role of educators is a unique contribution of the article that has not been discussed in the mixed methods literature to date. We also hope to uniquely map the contribution of authors at the beginning of mixed methods, and to take a current assessment of its growth. A focus on topics being discussed may enable other researchers to build on current streams of thinking and help advance the literature with their own studies. Reflecting on the various stances about the nature of mixed methods research should reinforce the idea that this form of inquiry is emerging and has not yet reached the consensus stage.

 

Developmental overview of the mixed methods field

It is a fair statement that mixed methods research has come into its own as a research approach in the last 20 years. Like bookends on a library shelf, we can map the early beginnings, as well as the current status of interest to see the modest start, as well as the substantial growth over time. To understand the beginnings of mixed methods, we need to return to the 1980s. Prior to this decade, authors wrote about the importance of gathering both quantitative and qualitative data (Jick, 1979) and debated the merits of combining qualitative and quantitative data (Reichardt & Cook, 1979). These early writers had not conceptualized mixed methods as a distinct approach to inquiry.

Several writers working independently in different parts of the world conceptualized mixed methods as we know it today. In 1988, in the US, two professors of sociology, John Brewer at Trinity College in Connecticut and Albert Hunter at Northwestern University, authored the first book looking at the synthesis of qualitative and quantitative data methods called Multimethod Research: A Synthesis of Styles. Also in 1988, a second book on mixed me- thods research would be issued, by a management, organizational specialist, Alan Bryman of the UK, titled Quantity and Quality in Social Research. In that same year, the educator and evaluator, Jennifer Greene at Cornell University and her colleagues, presented a paper on "mixed-method evaluation designs" at the Annual Meeting of the American Evaluation Association Meeting in New Orleans, Louisiana. This paper would be published in the journal Educational Evaluation and Policy Analysis the following year (Greene, Caracelli, & Graham, 1989) as a "Conceptual Framework for Mixed-Method Evaluation Designs. Also, beginning in 1989, in the US in Nebraska, John W. Creswell (senior author) was writing a chapter on "Combined Qualitative and Quantitative Designs" that would come out in 1994 in his book by Sage Publications titled, Research Design: Qualitative and Quantitative Approaches. The only specific writing on the research approach of mixed methods that he had to rely on was the work by Jennifer Greene and her colleagues (Greene, Caracelli & Graham, 1989). All of these writers had come to the same conclusion that it was timely to advance an approach to research that combined both quantitative and qualitative modes of inquiry.

In this development, we find that both Creswell and Greene were educators. Still more educators are found as authors of important studies that have provided the books for mixed methods research some 14 and still counting to date (see Tashakkori & Creswell, 2007) and the articles for discussion leading the path for mixed methods research (see Table 1 for an alphabetical listing of the authors from the field of education and their contributions, mentioned in this article). Books, of course, are only one way to assess the magnitude of developments in the field of mixed methods research.

 

 

Other markers of the development of mixed methods research include journals publishing mixed methods research, conferences including or focusing on mixed methods, specific courses on mixed methods, and funding agencies that are encouraging and awarding grants to mixed methods projects. Several journals are now available that are receptive to or exclusively publish mixed methods studies or discussions about mixed methods [Journal of Mixed Methods Research, Quality and Quantity, Field Methods, International Journal of Multiple Research Approaches (IJMRA)]. Examples of mixed methods studies abound in the literature, and with increasing frequency they are called "mixed methods" studies an improvement over past studies, which used words such as "qualitative and quantitative" or "combined" or "integrated" research. Interest can be found throughout the world in using mixed methods approaches by noting recent publications of studies from Sri Lanka (Nastasi, Hitchcock, Sarkar, Burkholder, Varjas & Jayasena, 2007), Germany (Bernardi, Keirn & von der Lippe, 2007), Japan (Fetters, Yoshioka, Greenberg, Gorenflo & Yeo, 2007), and the UK (O'Cathain, Murphy & Nicholl, 2007). A major conference on mixed methods research now is entering its fourth year at the University of Cambridge, UK, and new international conferences are on the horizon. A number of courses are now available on "mixed methods research" (Creswell, Tashakkori, Jensen & Shapley, 2003) and several on-line mixed methods courses are available (at the University of Nebraska-Lincoln taught by Ron Shope and at the University of Alabama-Birmingham by the educator, Nataliya Ivankova). Funding agencies encourage mixed methods research such as the National Institutes of Education in the US (National Institutes of Health, Office of Behavioral and Social Sciences, 1999) and in the UK (Bryman, 2006).

 

Issues in mixed methods research

One aspect of this worldwide "movement" is the clarification of and the clustering around topics by various scholars writing about mixed methods research. The field has emerged enough so that topics and individuals writing about them can be identified. The following discussion is not meant to be an exhaustive list of topics currently being discussed, but a discourse to encourage others to map this emerging field of research and add their contributions to growing discussions. Topics being discussed (in no apparent order) in the mixed methods community include the growth of a specific research language for mixed methods, mixed methods designs, how quantitative and qualitative data are to be integrated, the value and contributions of mixed methods research, explicit techniques of conducting mixed methods studies, the steps involved in carrying out mixed methods research, the tension between the paradigms in quantitative and qualitative research, the discussion of who is influencing and guiding the field of mixed methods research, and the basic question of what is mixed methods research.

One issue is the emergence of a language of research, a bilingual language. This language is neither quantitative nor qualitative. For example, Onwuegbuzie & Johnson (2006), two educators, call for the use of the term, "legitimacy," rather than validity.

Another issue is a continued discussion about the types of research designs available to conduct a mixed methods study. These range from a parsimonious set of designs from the educators Creswell and Plano Clark (2007) to complex, longitudinal evaluation designs from educational school psychologists, Nastasi and her colleagues (Nastasi et al., 2007).

Another topic that mixed methods researchers are discussing is how quantitative and qualitative data are "mixed" in a mixed methods study? This discussion goes from where in the process of research the data are mixed (e.g. data analysis, interpretation), how they may be mixed (e.g. one form of data are transformed into the other), to why they are mixed or, altematively, the reasons for mixing (see Bryman's, 2006 list of 16 reasons).

In addition, conversations are emerging about the value-added by mixed methods research. The question is whether mixed methods research provides "more" than quantitative or qualitative research alone (e.g. better understanding of the problem, more efficiency, more cost effective, more evidence, and so forth). The article by O'Cathain et al. (2007), for example, probes the "yield" from mixed methods studies in the health sciences in UK.

Ideas are also being exchanged about the "techniques" of conducting a mixed methods study. Issues in this domain include the unusual blends of data such as discourse analysis with structural equation modeling, the approaches available for selecting qualitative follow-up participants from a survey, strategies for resolving contradictions when researchers merge quantitative and qualitative datasets, and the specific uses of quantitative and qualitative software programs to assist mixed methods researchers (Creswell, 2008).

The practice of conducting a mixed methods study has also drawn the attention of mixed methods writers (Corden & Hirst, in press). Many questions may arise in implementing a mixed methods study that would have to be addressed. Some steps involved in implementing a mixed methods study include how key decisions are negotiated, the formation of teams of researchers and their dynamics of interaction, the skills that individuals bring to research teams, the authorship decisions among team members, and the logistics of conducting a mixed methods study as a "lone" researcher.

Another issue is the continued tension to co-exist between those philosophically inclined and those methodologically inclined. Concems that may arise in this context include whether researchers can use multiple worldviews and paradigms in their mixed methods study, how compatibility can be established on an academic team of mixed methods researchers, and whether "communities of practice" (Morgan, 2007; Descombe, in press) exist in which shared beliefs and values prevail within a field of study or in an interdisciplinary configuration of team members or informal networks of scholars.

From a postmodern perspective, there is an on-going dialogue about the "discourse" of mixed methods research. Who is controlling the discussion (Freshwater, 2007), the language used [Does it lean toward post-positivist and quantitative approaches? asks Howe (2004)]. The lack of agreement on definitions and the conflicts and confluence among different scholars from different nations is yet another area for dialogue.

Finally, there are continued discussions about what constitutes mixed methods research. This discussion spans from how it might be defined to stances groups of individuals take in this endeavour. It seems that it is the core issue on the table right now. Again, educators have been central to much of this discussion to which we now turn.

 

The nature of mixed methods research from multiple perspectives

As with all new methodologies, the early stages of development are marked by some confusion and debate about the nature of the methodology. Such has been the case with mixed methods research. At its basic elements, mixed methods research involves collecting both quantitative and qualitative data (in response to quantitative and qualitative research questions), the merging, linking, or combining of the two sources of data, and then conducting research as a single study or a longitudinal project with multiple phases. From the perspective of the editors of the Journal of Mixed Methods Research, we also add into this configuration a need for the mixed methods project to advance our understanding of this form of inquiry. Authors need to position their study within the on-going discussions about mixed methods research (Creswell & Tashakkori, 2007a).

The difficulty of conceptualizing mixed methods in this way is that we have a diverse community engaged in mixed methods research. Scholars come from many social and human sciences fields, such as sociology, psychology, marketing, education, communications studies, family medicine, and nursing, to name just a few areas. It is not surprising that views of what constitutes mixed methods research will vary. However, the different stances can be seen easily (see also Creswell & Tashakkori, 2007b) from those who focus on methods, on the process of research, on philosophical issues, and those who build on existing research designs.

First there are the applied methodologists who tend to focus on the research methods (see Brewer & Hunter, 1988; Creswell, 2003). Mixed methods, to them, means the collection, analysis, and interpretation (i.e. the methods) of both quantitative and qualitative data in a study. This approach forces a focus on the "techniques" of research, and it might be seen as a "clean" way to view mixed methods without being encumbered by philosophy or other aspects of the research process (e.g. the research questions). The disadvantage in this viewpoint is that the division between what constitutes quantitative data and qualitative data is not always clear. Quantitative data (e.g. scales on an attitudinal instrument) does tend to be more close-ended while qualitative data (e.g. focus-group discussion) are more open-ended. Student records at a school, for instance, may be viewed by a researcher as either quantitative information or qualitative information.

A second group of individuals view mixed methods not as a method, but as a process of research that encompasses all phases of the research process, not just the methods (see the educators, Tashakkori & Teddlie, 2003). Included within this conception would be the phases of research from the broad philosophical assumptions directing the inquiry to the final interpretation and report of the study. Their idea is to "mix" at all the stages of the research approach, quantitative, and qualitative research. The advantage of seeing mixed methods this way is that one cannot easily disentangle the methods from the research questions and the research questions from the larger guiding philosophical assumptions. Indeed, research follows a systems perspective of all parts that work in tandem throughout the process of inquiry. The difficulty with this perspective is that it then calls researchers to mix multiple paradigms or worldviews in a study (e.g. a post-positivist paradigm and a constructivist paradigm). Some writers have called this linking of different paradigms as "incompatible" (Smith & Heshusius, 1986; Sale, Lohfeld & Brazil, 2002). In their view, these philosophical assumptions that one brings to research, called "paradigms" or "beliefs", cannot be combined.

A third group of mixed methods scholars feel that the methods are incidental to inquiry and that the philosophical assumptions are the important focus of inquiry. This perspective about mixed methods has a strong hold in the Commonwealth countries where the tone toward research is less the empirical models of inquiry that have dominated the US and more toward a philosophical debate about approaches to research (see Giddings, 2006). The advantage of this approach is that few scholars would argue that a broad philosophical stance is a component of conducting research (whether it is expressed or implied in the written report) and that all inquirers bring to research views about issues such as the basis for claiming knowledge, the nature of reality, and the place of values in inquiry. This line of thinking, however, raises the important question as to what the nature of the philo- sophical assumptions might be. In the field of mixed methods, a number of scholars embrace pragmatism as the philosophical underpinning for conducting mixed methods research (Morgan, 2007; Tashakkori & Teddlie, 2003). Pragmatism is an American philosophical approach developed by writers such as Dewey, Meade, and James, and may not play well abroad today. Pragmatism, as discussed by the mixed methods writers means that the focus of research is on the research question and different methods can be employed to answer this question. Multiple, pluralistic approaches to research are all viable, and the emphasis is on "what works" (Tashakkori & Teddlie, 1998). Others argue for a philosophy based on advocacy or under-represented groups (Mertens, 2003), while still others have more of an eclectic approach to allowing multiple philosophies to inform mixed methods inquiry (Greene, 2007; Creswell & Plano Clark, 2007). The disadvantage of thinking about mixed methods from this framework is that it is not applied enough (see Morgan, 2007). A discussion about philosophy tells us little about how to conduct a mixed methods study, what dilemmas might emerge in doing so, or the practicalities of how to begin thinking about combining both quantitative and qualitative data in a study.

A recent trend to emerge is to think about mixed methods as a means of collecting, analysing, and using both qualitative and quantitative data within an established approach (or research design or methodology). This issue emerged for us when an individual at one of our workshops asked whether ethnography was mixed methods research. Ethnographers have typically collected both qualitative data (e.g. interviews) and quantitative data (e.g. surveys), and analysed both during a project (see the ethnographic data collection procedures recommended by LeCompte & Schensul, 1999). This trend toward linking mixed methods procedures with established research designs seems like a practice of mixed methods growing in importance. It would be valuable for mixed methods writers to start looking closely at the conduct of mixed methods studies (see Cordon & Hirst, in press) to see how mixed methods is being implemented. For example, researchers have developed a multiple case study in which both quantitative and qualitative data are gathered (see Luck, Jackson & Usher, 2006) and have conducted experiments in the health sciences with qualitative data flowing into an otherwise quantitative intervention trial before the treatment, while the treatment is being conducted, or after the treatment (Sandelowski, 1996), An entire book has been written by Jane Eliot on narrative research (Eliot, 2005) in which quantitative longitudinal designs are used alongside qualitative story gathering. The use of visual ethnography (Pink, 2001) also holds potential for combining in documentaries elements of quantitative trends with the qualitative words of participants.

 

Conclusion

The etiology of a research approach probably follows statements made by a few individuals early in the emergence of the idea, followed by a period of debate and discussion about direction, and the final emergence of a consen- sus approach that is widely embraced and becomes the standard for the field. As an emerging approach to inquiry, mixed methods has yet to reach consensus and seems to be in the debate and discussion stage of development. Discussing different stances on the nature of this inquiry marks the stage for mixed methods as an active debate. Mentioning specific topics moves in the direction of consensus, as well as "marking" the history of the concept, the educators involved, and its current interest.

Increased interest in mixed methods research, worldwide, is likely to continue over the next few years. Like qualitative research, we believe that mixed methods is here to stay as a research approach. Funding initiatives, available journals, and perhaps, most importantly, the students who gravitate to it will provide the needed momentum for it to continue to be a "movement". Perhaps it can be predicted that it will expand in scope geographically to more countries in the world and that it will continue to draw from scholars who have traditionally been trained in quantitative or qualitative research. In education, a field that has always been open to many possibilities and perhaps not locked into somewhat rigid disciplinary trends such as often found in the fields of sociology and psychology, the openness to experiment with research methodologies and ways of thinking about research will encourage mixed methods research. Already sub-fields such as counseling psychology, physics education, and leadership have begun to embrace mixed methods research (Klenke, in press; Plano Clark, 2005), and the educational sub-fields will undoubtedly place their individual stamp on the field. The text on mixed methods specifically written for educators has yet to be written, but it is on the horizon.

These predictions must be seen as looking into a crystal ball. However, the impact of mixed methods can be seen as a "movement". Developments have occurred in a reasonably short period of 20 years, involving many edu- cators in the process, and engaging the attention of scholars worldwide. The emergence of various perspectives about the nature of mixed methods research is in a healthy state of development.

 

References

Bergman M (ed.) 2008. Advances in mixed methods research: Theories and applications. London: Sage.        [ Links ]

Bernardi L, Keim S & von der Lippe H 2007. Social influences on fertility: A comparative mixed methods study in Eastern and Western Germany. Journal of Mixed Methods Research, 1:23-47.        [ Links ]

Brewer J & Hunter A 1989. Multimethod research: A synthesis of styles. Newbury Park, CA: Sage Publications.        [ Links ]

Bryman A 1988. Quantity and quality in social research. London: Routledge.         [ Links ]

Bryman A 2006. Integrating quantitative and qualitative research: How is it done? Qualitative Research, 6:97-113.        [ Links ]

Corden A & Hirst M (in press). Implementing a mixed methods approach to explore the financial implications of death of a life partner. Journal of Mixed Methods Research.        [ Links ]

Creswell JW 1994. Research design: Qualitative and quantitative approaches. Thousand Oaks, CA: Sage.        [ Links ]

Creswell JW 2003. Research design: Qualitative, quantitative, and mixed methods approaches, 2nd edn. Thousand Oaks, CA: Sage.        [ Links ]

Creswell JW 2008. Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Upper Saddle River, NJ: Pearson/Merrill Education.        [ Links ]

Creswell JW & Plano Clark VL 2007. Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.        [ Links ]

Creswell JW & Tashakkori A 2007a. Editorial: Developing publishable mixed methods manuscripts. Journal of Mixed Methods Research, 1:107-111.        [ Links ]

Creswell JW & Tashakkori A 2007b. Editorial: Differing perspective on mixed methods research. Journal of Mixed Methods Research, 1:303-308.        [ Links ]

Creswell JW, Tashakkori A, Jensen KD & Shapley KL 2003. Teaching mixed methods research: Practices, dilemmas, and challenges. In: Tashakkori A & Teddlie C (eds). Handbook of mixed methods in social and behavioral research, pp. 619-637. Thousand Oaks, CA: Sage.        [ Links ]

Descombe M (in press). Communities of practice: A research paradigm for the mixed methods approach. Journal of Mixed Methods Research.        [ Links ]

Eliot J 2005. Using narrative in social research: Qualitative and quantitative approaches. London: Sage.        [ Links ]

Fetters MD, Yoshioka T, Greenberg GM, Gorenflo DW & Yeo SA 2007. Advance consent in Japanese during prenatal care for epidural anesthesia during childbirth. Journal of Mixed Methods Research, 1:333-365.        [ Links ]

Freshwater D 2007. Reading mixed methods research: Contexts for criticism. Journal of Mixed Methods Research, 1:134-146.        [ Links ]

Giddings LS 2006. Mixed-methods research: Positivism dressed in drag? Journal of Research in Nursing, 11:195-203.        [ Links ]

Greene JC 2007. Mixed methods in social inquiry. San Francisco: Jossey-Bass.        [ Links ]

Greene JC & Caracelli VJ 1997. Advances in mixed methods evaluation. San Francisco: Jossey-Bass.        [ Links ]

Greene JC, Caracelli VJ & Graham WF 1989. Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11:255-274.        [ Links ]

Howe KR 2004. A critique of experimentalism. Qualitative Inquiry, 10:42-61.         [ Links ]

Jick TD 1979. Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24:602-611.        [ Links ]

Kemmis S & Wilkinson M 1998. Participatory action research and the study of practice. In: Atweh B, Kemmis S & Weeks P (eds). Action research in practice: Partnerships for social justice in education. London and New York: Routledge.        [ Links ]

Klenke K (in press). Qualitative research methods in the study of leadership. Amsterdam: Elsivier.        [ Links ]

Kline RB 1998. Principles and practice of structural equation modeling. New York: Gilford.        [ Links ]

LeCompte MD & Schensul JJ 1999. Designing and conducting ethnographic research (Ethnographer's toolkit, Vol. 1). Walnut Creek, CA: AltaMira.        [ Links ]

Luck L, Jackson D & Usher K 2006. Case study: A bridge across the paradigms. Nursing Inquiry, 13:103-109.        [ Links ]

Mertens DM 2003. Mixed methods and the politics of human research: The transformative-emancipatory perspective. In: Tashakkori A & Teddlie C (eds). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage.        [ Links ]

Morgan DL 2007. Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1:48-76.         [ Links ]

Nastasi BK, Hitchcock J, Sarkar S, Burkholder G, Varjas K & Jayasena A 2007. Mixed methods in intervention research: Theory to adaptation. Journal of Mixed Methods Research, 1:164-182.        [ Links ]

National Institutes of Health, Office of Behavioral and Social Sciences Research 1999. Qualitative methods in health research: Opportunities and considerations in application and review. Washington, DC: National Institutes of Health.        [ Links ]

O'Cathain A, Murphy E & Nicholl J 2007. Integration and publications as indicators of "yield" from mixed methods studies. Journal of Mixed Methods Research, 1:147-163.        [ Links ]

Onwuegbuzie AJ & Johnson RB 2006. The validity issue in mixed research. Research in the Schools, 13:48-63.         [ Links ]

Pink S 2001. Doing visual ethnography: Images, media, and representation in research. London: Sage.        [ Links ]

Plano Clark VL 2005. Cross-disciplinary analysis of the use of mixed methods in physics education research, counseling psychology, and primary care. Doctoral dissertation, University of Nebraska-Lincoln. Dissertation Abstracts International, 66, 02A.        [ Links ]

Plano Clark VL & Creswell JW 2008. The mixed methods reader. Thousand Oaks, CA: Sage.        [ Links ]

Reichardt CS & Cook TD 1979. Beyond qualitative versus quantitative methods. In: Cook TD & Reichardt CS (eds). Qualitative and quantitative methods in evaluation research. Beverly Hills, CA: Sage.        [ Links ]

Sale JEM, Lohfeld LH & Brazil K 2002. Revisiting the quantitative-qualitative debate: Implications for mixed-methods research. Quality and Quantity, 36:43-53.        [ Links ]

Sandelowski M 1996. Using qualitative methods in intervention studies. Research in Nursing & Health, 19:359-364.        [ Links ]

Smith JK & Heshusius L 1986. Closing down the conversation: The end of the quantitative-qualitative debate among educational inquirers. Educational Researcher, 15:4-12.        [ Links ]

Tashakkori A & Creswell JW 2007. The new era of mixed methods. Journal of Mixed Methods Research, 1:3-7.        [ Links ]

Tashakkori A. & Teddlie C 1998. Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage Publications.        [ Links ]

Tashakkori A & Teddlie C (eds) 2003. Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage.        [ Links ]