Qualitative Secondary Analysis in Austere Times: A Reply to Coltart, Henwood and Shirani

Abstract: In their article, published in this journal, COLTART, HENWOOD and SHIRANI raise a number of issues regarding the effective and ethical conduct of qualitative secondary analysis. In doing so they seek to exemplify general points about secondary analytic practice and ethics with reference to the UK Timescapes research programme in which they were involved as primary researchers and we were involved as secondary analysts. They position our work in ways we find unrecognisable, and potentially misleading. We briefly re-describe aspects of our work, and our key arguments, with reference to the timing of secondary analysis, knowledge claims and the contextual embeddedness of qualitative data.

Key words: qualitative secondary analysis; qualitative research ethics; Timescapes

In their article published in this journal COLTART, HENWOOD and SHIRANI (2013) raise a number of issues regarding the effective and ethical conduct of qualitative secondary analysis, drawing on their experience as primary researchers within Timescapes, a programme of qualitative longitudinal research funded by the Economic and Social Research Council in the UK. In their criticisms they have implicated the current authors, yet they paint a picture of our work which we do not recognise. We reject their suggestions that primary researchers become "data donors" to secondary analysts, and that primary and secondary researchers are in some kind of race to publish, and we do not see these as worthy of further discussion. We focus here instead on what we take as the three main areas of concern for COLTART and colleagues, and show briefly why their representation of our work is misleading. After a short description of Timescapes, we consider COLTART and colleagues' reflections on the timing of primary and secondary analysis, and the challenges of running these concurrently. Next we address their argument that we "over-privilege secondary analysis as a knowledge building strategy" (§28). Thirdly, COLTART and colleagues appear to find parallels between our work and a critiqued practice of "pooling disembedded data" (§29). We are stunned by this suggestion since throughout our work we offer a sustained engagement with the contextual embeddedness of data, and develop our analyses accordingly. We briefly re-describe our position and, in so doing, re-assert our argument of the possibility of a "middle way" for secondary analysis. [1]

Timescapes (Changing Lives and Times: Relationships and Identities through the Life Course) was funded as part of the ESRC Qualitative Longitudinal (QL) Initiative, and comprised a five year long programme of work centring on seven primary qualitative longitudinal research projects, the creation of a new qualitative longitudinal data archive, and a programme of secondary analysis activities. The primary research projects were run by teams in five different universities, from 2007 to 2011.¹⁾ The projects were independent, and some were in place before Timescapes commenced. They had in common substantive interests in biography, life course transitions, familial relationships and inter-generational dynamics. They were all qualitative longitudinal projects, and funded as part of the Timescapes programme. The seven projects have provided the data which are at the heart of the Timescapes QL archive. [2]

Timescapes included a range of secondary analysis activities which were integral to the overall programme from the start, and which included cross-project work undertaken by teams themselves, and a dedicated secondary analysis project (see IRWIN, BORNAT & WINTERTON, 2012 for an early overview). The latter project ran through 2010-2011. It was undertaken by Sarah who led the project and Mandy, the project Research Fellow, who was employed for eighteen months from April 2010. We liaised closely with Joanna who led the wider secondary analysis strand within Timescapes. Our practices and outputs are documented on the Timescapes website and include working papers and descriptions of how we proceeded (e.g. IRWIN & WINTERTON, 2011a), as well as substantive published articles (IRWIN, 2013; IRWIN & WINTERTON, 2012; IRWIN et al., 2012; WINTERTON & IRWIN, 2012). The secondary analysis project aims were explicit from the funding proposal stage onwards, and included working with data from across Timescapes primary projects. [3]

COLTART and colleagues point to particular difficulties arising from the concurrent timing of primary and secondary analysis. We agree that particular intellectual, ethical and practical issues arose in such a context. The overall Timescapes programme was complex and, with its different strands of primary research, "live" data archiving and secondary analysis activities, there were many challenges. All primary projects and the secondary analysis project were scheduled to have finished by 2011. In their article, COLTART and colleagues particularly emphasised what they saw as risks of concurrent secondary and primary analysis, unless these are undertaken as part of a mutual primary data sharing exercise. However it is important to be clear that, within the collective undertaking which was Timescapes, concurrency was a structural necessity: a condition of funding for the primary projects as well as the secondary analysis project. [4]

To be sure: there are important matters relating to the ethical and effective conduct of secondary analysis, and risks entailed in engaging with data whose moral ownership lies ultimately with the originator researchers and their participants. A concern for COLTART and colleagues arises from how they perceive epistemological ownership, the rights over how data might be interpreted. This points to some complex issues. Perhaps from the outset of collaborations between primary and secondary analysts, there need to be absolutely clear protocols in place regarding the use of data, the responsibilities of the primary and secondary analysts, and the rights and obligations of each. These are challenging enough when issues arising, and related interests and concerns, are known. Within Timescapes the challenges were sharpened since such issues arose and needed resolving within an evolving programme of research. We document (in IRWIN & WINTERTON, 2011a) our practices as secondary analysts in which we sought to anticipate and address some of these issues, including the production and circulation of an early discussion paper (subsequently edited into a working paper, IRWIN & WINTERTON, 2011b), holding meetings with each of the teams, and sharing final analyses in draft publication form.²⁾ Furthermore we note that this cross project work formed just one strand of our work and, within it, our published work in the area relates as much to questions of method as it does to substantive analysis. The secondary analysis project team worked in a context which pressed against a close and ongoing collaboration with primary researchers over analysis, but this was not part of our remit. Further, Timescapes entailed a range of models of secondary analysis and other members of the Timescapes programme undertook both concurrent and subsequent secondary analysis, producing working papers, published articles and book chapters.³⁾ In short, the timing of secondary analysis did create complex issues within an evolving programme of research, but we suspect the model of concurrency will remain a rather specific and unusual undertaking.⁴⁾ [5]

COLTART and colleagues take issue with an alleged "implicit suggestion" that secondary analysis has potential for explanation and theory building which outstrips primary analysis. This is a complete misrepresentation of our position. We very briefly rehearse our arguments, since we have elaborated these more fully elsewhere. COLTART and colleagues do not like our early position statement presented in a working paper (which they cite), and in the subsequent peer reviewed publication (IRWIN & WINTERTON, 2012). In both, we discussed debates about primary and secondary analysis, proximity to contexts of data production, and knowledge claims, and the extent to which the latter require a primary researcher positioning. Drawing on HAMMERSLEY (2010) we argued that, in respect of developing explanation, it may be more productive to reflect on divisions between data and evidence than on divisions between primary and secondary analysts. The emphasis on divisions between data and evidence encourages researchers to focus on the grounds on which knowledge claims are built, whether by primary or secondary analysts (cf. HAMMERSLEY, 2010). COLTART and colleagues are particularly incensed by our statement that "overplaying proximate context may privilege description over explanation. Grounding knowledge claims will often entail stepping outside the specifics of the data and relating it to our theories, and to other evidence" (IRWIN & WINTERTON, 2011c p.17). For COLTART and her colleagues:

"... as primary researchers we would take issue with the implicit suggestion that the distance afforded by secondary analysis (a distance which is seen to allow it to take in more data sets, perspectives and evidence) boosts opportunities to answer broader questions and develop theory. We would argue that this reflects a quantitative epistemological position (the myth of the omniscient researcher) which has been soundly critiqued" (§28). [6]

It is true that working with data from diverse data sets helped us to understand, and develop arguments, about the contextual embeddedness of data across different projects. However, there was no "implicit suggestion" that it is secondary analysts who have some special grasp on understanding. This is an imposition. We do maintain that overplaying proximate context may privilege description over explanation. This is certainly not to underplay the nuances of context. It is our belief that research requires continual reflection on the particulars of a set of research encounters enshrined in a data set, and the associated contexts, and broader or related bodies of evidence which enable comparison, contrast, and facilitate theorisation of process. We do so as primary researchers as well as former Timescapes secondary analysts. We were most certainly not advocating "... a unique role for QSA in terms of boosting the explanatory power of qualitative research by bridging proximate and distal contexts" (COLTART et al., §28). [7]

A third main plank of COLTART and colleagues' argument relates to the embeddedness of data. As they know, an important part of the philosophy of secondary analysis in Timescapes was to engage in some considerable detail with the contextual embeddedness of data (e.g. BISHOP, 2009; BORNAT, 2013; IRWIN & WINTERTON, 2012; IRWIN et al., 2012; McLEOD & THOMSON, 2009; SHELDON, 2009). This appears to be consistent with COLTART and colleagues' philosophy. However we are concerned that here, too, Sarah and Mandy are positioned as doing something very different, if not opposite, when the authors cite WEED's argument that "... pooling disembedded data sets in order to develop enhanced explanation is inconsistent with an interpretive epistemology" (WEED, 2005, cited in COLTART et al., §29). We would argue that "pooling disembedded data sets" is inconsistent with most epistemologies including any broadly realist one. COLTART and colleagues may not be critiquing our practice here but, since they cite no culprits, it must appear to readers that we are guilty of flying close to the risks they describe. We therefore summarise an example of cross-project secondary analysis work from the Timescapes Secondary Analysis project, since engaging with contextual nuance lay at the heart of our analyses. We saw the embeddedness of data as occurring at a number of levels, including ones relating to the immediate contexts of data production, and to the methodological shaping, and project specific embeddedness of data. The Timescapes projects themselves engaged with contexts in new and interesting ways, with diverse choices of method and new ways of exploring the relationships, and their temporal unfolding, which were often the focus of research. Since part of our task was to work across a subset of Timescapes data we needed to engage with the complexity of data which were produced in very different contexts. Engaging with issues of context was a very central part of our secondary analysis undertaking. [8]

We describe elsewhere how we arrived at a set of questions relating to gender and issues of time and work life balance, as a potentially productive area for working across different Timescapes data sets (IRWIN & WINTERTON, 2011b, IRWIN & WINTERTON, Forthcoming). We posit that the approach we developed in our analyses was in many ways the opposite of "pooling disembedded data sets". We explored evidence within two very different data sets which both had potentially interesting evidence relating to our questions about gender, care, work and time, specifically: "Work and Family Lives: The Changing Experience of Young Families"⁵⁾ and "Masculinities, Identities and Risk: Transition in the Lives of Men as Fathers"⁶⁾, the latter being run by Karen HENWOOD. The different disciplinary and conceptual interests, the different sampling decisions, gender composition of the data sets and methods used in the projects underlined the inappropriateness of treating data as separable from the conditions of their production. Rather than lay data from different projects side by side we sought to understand and analyse patterning and process within data sets, and only then explore any parallels across the data sets, doing so also with reference to how data related to other, external, evidence and conceptual questions around gender inequalities. We sought to bring evidence into comparison on the basis of translating our questions, and emergent hypotheses, across project contexts in such a way as to enable an analytic conversation across the data sets. Engaging critically with specificity and the contextual embeddedness of data lay at the heart of our secondary analysis project, and was a position we advocated throughout. [9]

COLTART and colleagues' understanding of what is meant by collaborative working is one of a number of models which were used within Timescapes.⁷⁾ Any attempt to analyse data "as an outsider" they seem to see as disembedding data, unless it is done as part of a team effort working very closely and in ongoing interaction with the primary researchers. In our own practices we were never seeking to "edge" secondary analysis but rather develop a workable strategy in which we explored the possibilities for working across data sets, as secondary analysts, and we drew fairly strong conclusions about constraint and complexity in so doing. It may be that COLTART and colleagues feel that secondary analysis cannot be undertaken without the central involvement of the primary researcher. If this is the case then they are confining secondary analysis to a narrow range of practices and going against a tide of research running in a very different direction. Researchers turn to archived data for diverse reasons. We would ask: if data may only be revisited in the company of the primary researcher will this not constrain enquiry and ossify the original material? [10]

2) We note that COLTART and colleagues express concern that "professional and ethical challenges posed by QSA [qualitative secondary analysis] have sometimes been pushed towards the background ... or viewed as less vexing than previously thought" (§1), and we would suggest that our own practice with respect to the primary research teams, in terms of consultation regarding emergent analyses and publications has been sensitive to this. The response to COLTART et al. may have been different or even unnecessary, had we been given the opportunity to offer our comments on their paper, prior to submission. Thus we could not agree more that QSA does indeed raise ethical issues, but all parties may be the object of these concerns as professional relationships are being re-negotiated in the context of QSA. <back>

4) We note that there is growing interest in the area, for example see Changing Landscapes of the Third Sector (http://changinglandscapes.leeds.ac.uk/ [Accessed: October 1, 2013]).

5) "Work and Family Lives: The Changing Experiences of Young Families" was directed by Professor Kathryn BACKETT-MILBURN at the University of Edinburgh. We are grateful to Kathryn and her team for providing us with access to the" Work and Family Lives" data, and for having a dedicated project meeting with us in winter 2010-11. We are also grateful to Dr. Jeni HARDEN for her feedback on our use of "Work and Family Lives" data. This does not mean she necessarily concurs with our analysis. <back>

6) "Masculinities, Identities and Risk: Transition in the Lives of Men as Fathers" was directed by Professor Karen HENWOOD at the University of Cardiff. We are grateful to Karen and her team for providing us with access to "Men as Fathers" data from interviews conducted in East Anglia from 2000-2008, contextual information regarding the work arrangements of participants and their partners, and for having a dedicated project meeting with us in winter 2010-11. We are also grateful to Dr. Fiona SHIRANI for her feedback on our use of "Men as Fathers" data. This does not mean she necessarily concurs with our analysis. <back>

7) Several collaborations were organised between participating projects during the Timescapes programme. <back>

Bishop, Libby (2009). Ethical sharing and re-use of qualitative data. Australian Journal of Social Issues, 44(3), 255-272.

Bornat, Joanna (2013). Secondary analysis in reflection: Some experiences of re-use from an oral history perspective. Families, Relationships and Societies, 2(2), 309-317.

Coltart, Carrie; Henwood, Karen & Shirani, Fiona (2013). Qualitative secondary analysis in austere times: Ethical professional and methodological considerations. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 14(1), Art. 18, http://nbn-resolving.de/urn:nbn:de:0114-fqs1301181 [Accessed: March 27, 2013].

Hammersley, Martyn (2010). Can we re-use qualitative data via secondary analysis? Notes on some terminological and substantive issues. Sociological Research Online, 15(1), 5. http://socresonline.org.uk/15/1/5.html [Accessed: February 1, 2011].

Irwin, Sarah (2013). Qualitative secondary data analysis: Ethics, epistemology and context. Progress in Development Studies, 13(4), 295-306.

Irwin, Sarah & Winterton, Mandy (2011a). Qualitative secondary analysis in practice: An extended guide (with reference to concepts, contexts and knowledge claims). Timescapes Working Paper 7, http://www.timescapes.leeds.ac.uk/resources/publications [Accessed: December 6, 2013].

Irwin, Sarah & Winterton, Mandy (2012). Qualitative secondary analysis and social explanation. Sociological Research Online, 17, 2, 4 http://socresonline.org.uk/17/2/4.html [Accessed: December 6, 2013].

Irwin, Sarah & Winterton, Mandy (Forthcoming). Gender and work-family conflict: A secondary analysis of Timescapes data, In Janet Holland & Rosalind Edwards (Eds.), Understanding families over time. London: Palgrave.

Irwin, Sarah; Bornat, Joanna & Winterton, Mandy (2012). Timescapes secondary analysis: Comparison, context and working across data sets. Qualitative Research, 12(1), 66-80.

McLeod, Julie & Thomson, Rachel (2009). Researching social change: Qualitative approaches. London: Sage.

Winterton, Mandy & Irwin, Sarah (2012). Teenage expectations of going to university: The ebb and flow of influences from 14 to 18. Journal of Youth Studies, 15(7), 858-874.

Sarah IRWIN (corresponding author) is a Reader in sociology, University of Leeds, and Director of the Centre for Research on Family, the Life Course and Generations. She was Director of the Timescapes Secondary Analysis project. She has researched and published extensively in the areas of family, parenting, youth, social inequalities and research methodology.

Joanna BORNAT is Emeritus Professor at the Open University. She was a member of the Timescapes Management Team and co-ordinator of the programme's secondary analysis strand. She has researched and published in the areas of oral history, migration, women's history, family change and gerontology.

Faculty of Health and Social Care
The Open University
Walton Hall
Milton Keynes, MK7 6AA, UK

Mandy WINTERTON is a lecturer in sociology at Edinburgh Napier University, and is the Director of the Military Research Centre at that institution. Mandy was the research fellow for the Timescapes Secondary Analysis project at the University of Leeds. Mandy's research and publications are in the areas of social class inequalities in education, social inclusion and transitions of military populations, and research methodology.

Life, Sport and Social Sciences
Edinburgh Napier University
Edinburgh EH11 4BN, UK

Irwin, Sarah; Bornat, Joanna & Winterton, Mandy (2014). Qualitative Secondary Analysis in Austere Times: A Reply to Coltart, Henwood and Shirani [10 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 15(1), Art. 23,
http://nbn-resolving.de/urn:nbn:de:0114-fqs1401231.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Commons Attribution 4.0 International License