Mixed Methods and Their Pragmatic Approach: Is There a Risk of Being Entangled in a Positivist Epistemology and Methodology? Limits, Pitfalls and Consequences of a Bricolage Methodology
Since the early 2000s, the pragmatic approach has been proposed as a philosophical program for social research, regardless of whether qualitative, quantitative or mixed methods are used. In addition, current mixed methods have been presented as a third way between positivism and constructivism. However, can mixed methods be fully considered a third way? For instance, in their inquiries, will scholars oriented to pragmatism actually employ the traditional and standardized questionnaire, with forced choices and closed questions, which strongly limits any interpretative and interactional perspective? Hence, several theoretical and methodological difficulties of the pragmatist proposal emerge precisely (and paradoxically) at the level of research practice. The pragmatic approach is presented by its proponents as a model designed to dissolve differences and neutralize epistemological barriers; however, without problematizing and removing the positivist features of their methods, researchers oriented to pragmatism actually risk ending up reproducing positivism in disguise. Hence, despite their claims to innovation, proponents of pragmatism are often overly traditionalist in their use of methods.
Amaturo, Enrica & Punziano, Gabriella (2016). I mixed methods nella ricerca sociale [Mixed methods in the social sciences]. Rome: Carocci.
Bazeley, Patricia (2018). Integrating analyses in mixed methods research. London: Sage.
Becker, Howard S. (1970). Sociological work: Method and substance. New Brunswick, NJ: Transaction Books.
Belli, Robert F.; Stafford, Frank & Alwin, Duane F. (2009). Calendar and time diary methods in life course research. Thousand Oaks, CA: Sage.
Biesta, Gert (2010). Pragmatism and the philosophical foundations of mixed methods research. In Abbas Tashakkori & Charles Teddlie (Eds.), Sage Handbook of mixed methods research social & behavioral research (pp.95-118). Thousand Oaks, CA: Sage.
Blumer, Herbert (1956). Sociological analysis and the "variable". American Sociological Review, 21(6), 683-690.
Bourdieu, Pierre; Chamboredon, Jean-Claude & Passeron, Jean-Claude (1968). Le métier de sociologue. Préalables épistémologiques [The craft of sociology. Epistemological preliminaries]. Paris: Mouton.
Brannen, Julia (1992). Mixing methods: Qualitative and quantitative research. London: Gower.
Bryman, Alan (2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1(1), 8-22.
Campbell, Donald T. & Fiske, Donald W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105.
Cicourel, Aaron V. (1964). Method and measurement in sociology. New York, NY: Free Press.
Cicourel, Aaron V. (1996). Ecological validity and White room effects. Pragmatics and Cognition, 4(2), 221-264.
Cini, Marcello (1994). Un paradiso perduto. Dall'universo delle leggi naturali al mondo dei processi evolutivi [Paradise lost. From the universe of natural laws to the world of evolutionary processes]. Milan: Feltrinelli.
Clinard, Marshall B. (1966). The sociologist's quest for respectability. The Sociological Quarterly, VII, 399-412.
Coats, Alfred W. (1989). Explanations in history and economics. Social Research, LVI(2), 331-360.
Collins, Kathleen M.T.; Onwuegbuzie, Anthony J. & Jiao, Qun G. (2007). A mixed methods investigation of mixed methods sampling designs in social and health science research. Journal of Mixed Methods Research, 1(3), 267-294.
Collins, Randall (1988). Theoretical sociology. San Diego, CA: Harcourt Brace Jovanovich.
Converse, Jean M. & Schuman, Howard (1974). Conversations at random: Survey research as interviewers see it. New York, NY: Wiley.
Creswell, John W. (2009). Research design. Qualitative, quantitative and mixed methods approaches. Thousand Oaks, CA: Sage.
Creswell, John W. & Plano Clark, Vicki L. (2017). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.
Cronbach, Lee J. (1975). Beyond the two disciplines of scientific psychology. American Psychologist, 30(2), 116-127.
Daigneault, Pier-Marc & Jacob, Steve (2014). Unexpected but most welcome: Mixed methods for the validation and revision of the participatory evaluation measurement instrument. Journal of Mixed Methods Research, 8(1), 6-24.
Denzin, Norman K. & Lincoln, Yvonna S. (2000). The discipline and practice of qualitative research. In Norman K. Denzin & Yvonna S. Lincoln (Eds.), The Sage handbook of qualitative research (pp.1-32). Thousand Oaks, CA: Sage.
Denzin, Norman K. & Lincoln, Yvonna S. (2005). Introduction: The discipline and practice of qualitative research. In Norman K. Denzin & Yvonna S. Lincoln (Eds.), The Sage handbook of qualitative research (pp.1-32). Thousand Oaks, CA: Sage.
Diesing, Paul R. (1971). Patterns of discovery in the social sciences. Chicago, IL: Aldine-Atherton.
Duncan, Otis D. (1984). Notes on social measurement. Historical and critical. New York, NY: Russell Sage Foundation.
Durkheim, Émile (1895). Les règles de la méthode sociologique [The rules of sociological method]. Paris: Payot.
Fakis, Apostolos; Hilliam, Rachel; Stoneley, Helen & Townend, Michael (2014). Quantitative analysis of qualitative information from interviews: A systematic literature review. Journal of Mixed Methods Research, 8(2), 139-161.
Flick, Uwe (2017). Mantras and myths: The disenchantment of mixed-methods research and revisiting triangulation as a perspective. Qualitative Inquiry, 23(1), 46-57.
Fowler, Floyd J. & Mangione, Thomas W. (1990). Standardized survey interviewing. Minimizing interviewer-related error. London: Sage.
Gardner, Howard (1983). Frames of mind. The theory of multiple intelligences. London: Paladin Books.
Giddings, Lynne S. (2006). Mixed-methods research: positivism dressed in drag. Journal of Research in Nursing, 11(3), 195-203.
Glaser, Barney G. & Strauss, Anselm L. (1967). The discovery of grounded theory. Chicago, IL: Aldine.
Gobo, Giampietro (1993). Le forme della riflessività. Da costrutto epistemologico a practical issue [The forms of reflexivity. From epistemological construct a practical issue]. Studi di Sociologia, 31(3), 299-317.
Gobo, Giampietro (2006). Set them free. Improving data quality by broadening interviewer's task. International Journal of Social Research Methodology: Theory & Practice, 9(4), 279-301, https://www.tandfonline.com/doi/full/10.1080/13645570600916064 [Accessed: January 16, 2023].
Gobo, Giampietro (2008). Re-conceptualizing generalization. Old issues in a new frame. In Alasuutari Pertti, Julia Brannen & Leonard Bickman (Eds.), The Sage handbook of social research methods (pp.193-213). London: Sage.
Gobo, Giampietro (2011). Back to Likert. Towards a conversational survey. In Malcolm Williams & Paul Vogt (Eds.), The Sage handbook of innovation in social research methods (pp.228-248). London: Sage.
Gobo, Giampietro & Mauceri, Sergio (2014). Constructing survey data. An interactional approach. London: Sage.
Gobo, Giampietro; Fielding, Nigel G.; La Rocca, Gevisa & van der Vaart, Wander (2022). Merged methods: A rationale for full integration. London: Sage.
Goffman, Erving (1967). Interaction ritual: Essays in face-to-face behavior. Chicago, IL: Aldine.
Gomm, Roger; Hammersley, Martin & Foster, Peter (Eds.) (2000). Case study method. London: Sage.
Gorard, Stephen (2010). Research design, as independent of methods. In Abbas Tashakkori & Charles Teddlie (Eds.), Sage Handbook of mixed methods in social & behavioral research (pp.237-251). Thousand Oaks, CA: Sage.
Greene, Jennifer C. (2007). Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass.
Greene, Jennifer C.; Caracelli, Valerie J. & Graham, Wendy F. (1989). Toward a conceptual framework for mixed method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255-274.
Hall, Jori N. (2013). Pragmatism, evidence, and mixed methods evaluation (Special Issue: Mixed methods and credibility of evidence in evaluation). New Directions for Evaluation, 138(2), 15-26.
Hammersley, Martyn (1987). Some notes on the terms "validity" and "reliability". British Educational Research Journal, 13(1), 73-81.
Hammersley, Martyn (2008). Troubles with triangulation. In Manfred Max Bergman (Ed.), Advances in mixed methods research (pp.22-36). London: Sage.
Hammersley, Martyn (2010). Is social measurement possible, and is it necessary?. In Geoffrey Walford, Eric Tucker & Madhu Viswanathan (Eds.), The Sage handbook of measurement (pp.409-426). London: Sage.
Hammersley, Martyn (2018). Commentary—on the "indistinguishability thesis": A response to Morgan. Journal of Mixed Methods Research, 12(3), 256-261.
Hesse-Biber, Sharlene (2015). Mixed methods research: The "thing-ness" problem. Qualitative Health Research, 25(6), 775-788.
Houtkoop-Steenstra, Hanneke (2000). Interaction and the standardized survey interview. The living questionnaire. Cambridge: Cambridge University Press.
Howe, Kenneth R. (1988). Against the quantitative-qualitative incompatibility thesis or dogmas die hard. Educational Researcher, 17, 10-16.
Howe, Kenneth R. (2004). A critique of experimentalism. Qualitative Inquiry, 10(4), 42-61.
Howell Smith, Michelle C.; Babchuk, Wayne A.; Stevens, Jared; Garrett, Amanda L.; Wang, Sherry C. & Guetterman, Timothy C. (2020). Modeling the use of mixed methods-grounded theory: Developing scales for a new measurement model. Journal of Mixed Methods Research, 14(2), 184-206, https://doi.org/10.1177/1558689819872599 [Accessed: January 16, 2023].
Jick, Todd D. (1979). Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24(4), 602-611.
Johnson, Burke R. & Onwuegbuzie, Anthony J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26.
Johnson, Burke R.; Onwuegbuzie, Anthony J. & Turner, Lisa A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2),112-133.
Kaplan, Abraham (1964). The conduct of inquiry: Methodology for behavioral science. San Francisco, CA: Chandler.
Karasz, Alison & Singelis, Theodore M. (2009). Qualitative and mixed methods research in cross-cultural psychology. Journal of Cross-Cultural Psychology, 40(6), 909-916.
Kelle, Udo (2001). Sociological explanations between micro and macro and the integration of qualitative and quantitative methods. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 2(1), Art. 5, https://doi.org/10.17169/fqs-2.1.966 [Accessed: August 3, 2022].
Kincheloe, Joe L. (2001). Describing the bricolage: Conceptualizing a new rigor in qualitative research. Qualitative Inquiry, 11(3), 679-692, https://link.springer.com/chapter/10.1007/978-94-6091-397-6_15 [Accessed: January 16, 2023].
Knorr Cetina, Karin (1999). Epistemic cultures: How the sciences make knowledge. Cambridge, MA: Harvard University Press.
Krontoft, Anna; Fuglsang, Lars & Kronborg, Hanne (2018). Innovation activity among nurses: The translation and preliminary validation of the Bricolage Measure—a mixed-method study. Nordic Journal of Nursing Research, 38(3), 151-159, https://journals.sagepub.com/doi/full/10.1177/2057158517733931 [Accessed: January 16, 2023].
Lazarsfeld, Paul Felix (1958). Evidence and inference in social research. Daedalus, 87(4), 99-130.
Lévi-Strauss, Claude (1962). La Pensée sauvage [The savage mind]. Paris: Plon.
Lewin, Kurt (1935). A dynamic theory of personality. New York, NY: McGraw-Hill.
Lieberson, Stanley (1985). Making it count. Berkeley, CA: University of California Press.
Likert, Rensis (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 1-55.
Lincoln, Yvonna S. & Guba, Egon G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage
Lundberg, George A. (1938). The concept of law in the social sciences. Philosophy of Science, V(2), 189-203.
Lundberg, George A. (1947). Can science save us?. New York, NY: Longmans, Green and Co.
Luyt, Russell (2012). A framework for mixing methods in quantitative measurement development, validation, and revision: A case study. Journal of Mixed Methods Research, 6(4), 294-316.
Mackenzie, Brian D. & Mackenzie, S. Lynne (1974). The case for a revised systematic approach to the history of psychology. Journal for the History of the Behavioural Sciences, X(3), 324-347.
Marradi, Alberto (1981). Misurazione e scale: Qualche riflessione e una proposta [Measurement and scales: Some reflections and a proposal]. Quaderni di Sociologia, 29(4), 595-639.
Marradi, Alberto (1985). Unità di misura e unità di conto [Units of measurement and units of counting]. Rassegna Italiana di Sociologia, 24(2), 229-238.
Maxcy, Spenser (2003). Pragmatic threads in mixed methods research in the social sciences: The search for multiple modes of inquiry and the end of the philosophy of formalism. In Abbas Tashakkori & Charles Teddlie (Eds.), Handbook of mixed methods in social & behavioral research (pp.51-89). Thousand Oaks, CA: Sage.
Maxwell, Joseph A. (2012). The importance of qualitative research for causal explanation in education. Qualitative Inquiry, 18(8), 655-661.
Maxwell, Joseph A. (2016). Expanding the history and range of mixed methods research. Journal of Mixed Methods Research, 10(1), 12-27.
Maxwell, Joseph A. (2018). The "silo problem" in mixed methods research. International Journal of Multiple Research Approaches, 10(1), 317-327.
Maxwell, Joseph A.; Chmiel, Margaret & Rogers, Sylvia (2015). Designing integration in mixed method and multi-method research. In Sharlene Nagi Hesse-Biber & Burke R. Johnson (Eds.), Oxford handbook of multimethod and mixed methods research inquiry (pp.223-239). New York, NY: Oxford University Press.
McKinney, John C. (1966). Constructive typology and social theory. New York, NY: Appleton-Century-Crofts.
Meehan, Eugene J. (1968). Explanation in social science. A system paradigm. Homewood: Dorsey Press.
Mokrzycki, Edmund (1983). Philosophy of science and sociology. From the methodological doctrine to research practice. London: Routledge & Kegan Paul.
Moreno, Jacob L. (1951). Sociometry, experimental method and the science of society. Ambler: Beacon House.
Morgan, David L. (2007). Paradigms lost and pragmatism regained. Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48-76.
Morgan, David L. (2014). Pragmatism as a paradigm for social research. Qualitative Inquiry, 20(8), 1045-1053.
Morgan, David L. (2018). Living within blurry boundaries: The value of distinguishing between qualitative and quantitative research. Journal of Mixed Methods Research, 12(3), 268-279.
Morse, Janice M. (2005). Evolving trends in qualitative research: Advances in mixed methods designs. Qualitative Health Research, 15(5), 583-585, https://doi.org/10.1177/1049732305275169 [Accessed: January 16, 2023].
Murdock, George P. (1949). Social structure. New York, NY: Macmillan.
Nadel, Siegfried Frederick (1951). The foundations of social anthropology. London: Cohen & West.
Nelson, Cary; Treichler, Paula & Grossberg, Lawrence (1992). Cultural studies: An introduction. In Lawrence Grossberg, Cary Nelson & Paula Treichler (Eds.), Cultural studies (pp.1-16). New York, NY: Routledge.
O'Cathain, Alicia; Murphy, Elizabeth & Nicholl, Jon (2007). Integration and publications as indicators of "yield" from mixed methods studies. Journal of Mixed Methods Research, 1(2), 147-153.
Onwuegbuzie, Anthony J. & Leech, Nancy L. (2005). On becoming a pragmatic researcher: The importance of combining quantitative and qualitative research methodologies. International Journal of Social Research Methodology, 8(5), 375-387.
Onwuegbuzie, Anthony J. & Collins, Kathleen M.T. (2007). A typology of mixed methods sampling designs in social science research. The Qualitative Report, 12(2), 281-316, https://doi.org/10.46743/2160-3715/2007.1638 [Accessed: January 16, 2023].
Parisi, Domenico & Castelfranchi, Cristiano (1978). Una definizione della psicologia cognitivista [A definition of cognitive psychology]. In Gaetano Kanizsa & Paolo Legrenzi (Eds.), Psicologia della gestalt e psicologia cognitivista [Gestalt psychology and cognitive psychology] (pp.63-84). Bologna: Il Mulino.
Payne, Geoff & Williams, Malcolm (2005). Generalization in qualitative research. Sociology, 39(2), 295-314.
Pearce, Lisa D. (2012). Mixed methods inquiry in sociology. American Behavioral Scientist, 56(6), 829-848.
Peters, Richard S. (1958). The concept of motivation. London: Routledge & Kegan Paul.
Quételet, Adolphe (1835). Sur l'homme et le développement de ses facultés, ou Essai de la physique sociale [A treatise on man and the development of his faculties, or Essay on social physics]. Paris: Bachelier.
Poth, Cheryl N. (2018). Innovation in mixed methods research. London: Sage.
Radnitzky, Gerard (1968). Contemporary schools of metascience. Goeteborg: Akademiforlaget.
Ragin, Charles (1987). The comparative method: Moving beyond qualitative and quantitative strategies. Berkeley, CA: University of California Press.
Ravindra, Raj (1975-1976). Experiment and experience: A critique of modern scientific knowing. Dalhousie Review, LV(4), 655-674.
Reichenbach, Hans (1930). Die philosophische Bedeutung der modernen Physik. Erkenntnis, 1(1), 49-71.
Riesman, David (1958). Some observations on the interviewing in the teacher apprehension study. In Paul F. Lazarsfeld & Thielens Wagner (Eds.), The academic mind (pp.266-370). Glencoe, IL: Free Press.
Runciman, Walter G. (1963). Social science and political theory. Cambridge: Cambridge University Press.
Sandelowski, Margarete (2014). Unmixing mixed methods. Research in Nursing and Health, 37(1),3-8, https://onlinelibrary.wiley.com/doi/full/10.1002/nur.21570 [Accessed: January 16, 2023].
Sandelowski, Margarete; Voils, Corrine I. & Knafl, George (2009). On quantitizing. Journal of Mixed Methods Research, 3(3), 208-222.
Sartori, Giovanni (1970). Concept misformation in comparative politics. American Political Science Review, 64(4), 1033-1053.
Schober, Michael F. & Conrad, Frederick G. (1997). Does conversational interviewing reduce survey measurement errors? Public Opinion Quarterly, 49, 576-602.
Schober, Michael F.; Conrad, Frederick G. & Fricker, Scott S. (2004). Misunderstanding standardized language in research interviews. Applied Cognitive Psychology, 18, 169-188, https://onlinelibrary.wiley.com/doi/abs/10.1002/acp.955 [Accessed: January 16, 2023].
Schreier, Margrit (2017). Contexts of qualitative research: Arts-based research, mixed methods, and emergent methods. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 18(2), Art. 6, https://doi.org/10.17169/fqs-18.2.2815 [Accessed: August 3, 2022].
Sedoglavich, Vesna; Akoorie, Michèle E. M. & Pavlovich, Katheyn (2015). Measuring absorptive capacity in high-tech companies: Mixing qualitative and quantitative methods. Journal of Mixed Methods Research, 9(3), 252-272.
Sharp, Heather (2019). Bricolage research in history education as a scholarly mixed-methods design. History Education Research Journal, 16(1), 50-62.
Shepard, Roger N. (1966). Metric structures in ordinal data. Journal of Mathematical Psychology, III(2), 287-315.
Sieber, Sam D. (1973). The integration of fieldwork and survey methods. American Journal of Sociology, 78(6), 1335-1359.
Silverman, David (2005). Instances or sequences? Improving the state of the art of qualitative research. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 6(3), Art. 30, https://doi.org/10.17169/fqs-6.3.6 [Accessed: August 3, 2022].
Silverman, David (2017). How was it for you? The interview society and the irresistible rise of the (poorly analysed) interview. Qualitative Research, 17(2), 144-158.
Skinner, Burrhus F. (1953). Science and human behaviour. New York, NY: Free Press.
Smith, Herman W. (1975). Strategies of social research: The methodological imagination. Englewood Cliffs, NJ: Prentice Hall.
Smythe, Elizabeth (2005). The thinking of research. In Pamela Ironside (Ed.), Beyond method: Philosophical conversations in healthcare research and scholarship (pp.223-258). Madison, WI: The University of Wisconsin Press.
Sorokin, Pitirim A. (1956). Fads and foibles in modern sociology and related sciences. Chicago, IL: Henry Regnery.
Spencer, Herbert (1862). First principles of a new system of philosophy. New York, NY: Appleton.
Stevens, Stanley S. (1946). On the theory of scales of measurement. Science, 103(2684), 677-680.
Strauss, Anselm & Corbin, Juliet (1990). Basics of qualitative research. London: Sage.
Tashakkori, Abbas & Teddlie, Charles (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage.
Tashakkori, Abbas & Teddlie, Charles (2010). Overview of contemporary issues in mixed methods research. In Abbas Tashakkori & Charles Teddlie (Eds.), Sage Handbook of mixed methods research in social & behavioral research (2nd ed., pp.1-44). Thousand Oaks, CA: Sage.
Taylor, Charles (1964). The explanation of behaviour. London: Routledge & Kegan Paul.
Teddlie, Charles &Tashakkori, Abbas (2003). Major issues and controversies in the use of mixed methods in social and behavioral sciences. In Abbas Tashakkori & Charles Teddlie (Eds.), Sage Handbook of mixed methods research in social & behavioral research (pp.3-50). Thousand Oaks, CA: Sage.
Teddlie, Charles & Yu, Fen (2007). Mixed methods sampling. A typology with examples. Journal of Mixed Methods Research, 1(1), 77-100.
Torgenson, Warren S. (1958). Theory and methods of scaling. New York, NY: Wiley.
Urban, Jennifer B.; Burgermaster, Marissa; Archibald, Thomas & Byrne, Alyssa (2015). Relationships between quantitative measures of evaluation plan and program model quality and a qualitative measure of participant perceptions of an evaluation capacity building approach. Journal of Mixed Methods Research, 9(2), 154-177.
Velleman, Paul F. & Wilkinson, Leland (1993). Nominal, ordinal, interval, and ratio typologies are misleading. The American Statistician, 47(1), 65-72.
Vidich, Arthur J. & Shapiro, Gilbert (1955). A comparison of participant observation and survey data. American Sociological Review, 20(1), 28-33.
Vogt, Paul W. (2008). Quantitative versus qualitative is a distraction: Variations on a theme by Brewer & Hunter (2006). Methodological Innovations Online, 3(1), 1-10, https://journals.sagepub.com/doi/pdf/10.4256/mio.2008.0007 [Accessed: January 16, 2023].
Webb, Eugene J.; Campbell, Donald Thomas; Schwartz, Richard D. & Sechrest, Lee (1966). Unobtrusive measures. Nonreactive research in the social sciences. Chicago, IL: Rand McNally.
Wheeldon, Johannes (2010). Mapping mixed methods research: Methods, measures, and meaning. Journal of Mixed Methods Research, 4(2), 87-102.
Yardley, Ainslie (2008). Piecing together—a methodological bricolage. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 9(2), Art. 31, https://doi.org/10.17169/fqs-9.2.416 [Accessed: August 3, 2022].
Yardley, Ainslie (2014). Children describing the world: Mixed-method research by child practitioners developing an intergenerational dialogue. Educational & Child Psychology, 31(1), 48-62.
Yee, Joyce S.R. & Bremner, Craig (2011). Methodological bricolage: What does it tell us about design?. Conference paper, Doctoral Education in Design Conference, May 23-25, Hong Kong, http://nrl.northumbria.ac.uk/8822/ [Accessed: August 3, 2022].
Copyright (c) 2023 Giampietro Gobo
This work is licensed under a Creative Commons Attribution 4.0 International License.