Volume 21, No. 1, Art. 19 – January 2020

Qualitative Content Analysis in Clinical Psychology to Explain the Pathomechanism of Personality Disorders

Emilia Soroko & Michał Dolczewski

Abstract: Qualitative content analysis (QCA) is a method for systematically describing qualitative data with the use of rule-bound procedures and human-generated interpretation. With this article we aim to reflect on the methodical aspect of QCA in the field of clinical psychology. After introducing the implementation of QCA procedures in clinical psychology and the diversity of content analytic-informed studies encountered in this field with special attention to coding frames, we present the problems and challenges that arise when using QCA in a more specific clinical context, namely, explanatory assessment of personality disorders. Beginning with the challenges (detecting clinically relevant information, explaining the pathomechanism, access to unconscious internal processes, role of relational information in assessment), we give possible solutions for the stage of data preparation and analysis. These include the balanced use of psychological theory and identifying patterns in data and in categories for explanation purposes; momentary process analysis aimed at assessing the dynamics of psychological life (intrapsychic and interpersonal); and relational context obtained from reflection on coding as a human process.

Key words: qualitative content analysis; clinical psychology; personality disorders; clinical explanatory assessment

Table of Contents

1. Introduction

2. Qualitative Content Analysis in Clinical Psychology

2.1 The place of QCA in the field of clinical psychology

2.2 Methodical diversity of studies using QCA in clinical psychology

3. Qualitative Content Analysis in Clinical Assessment of Personality Disorders

3.1 The specificity of clinical assessment of personality disorders

3.2 The role of QCA in the explanatory assessment of personality disorders

3.2.1 Detecting clinically relevant information—balancing theory and data

3.2.2 In search of explanatory power—patterns in and between categories

3.2.3 Dealing with unconscious and evolving speech—momentary processual analysis

3.2.4 Inclusion of relational context—interviewers and coders as sources of data

4. Summary

References

Authors

Citation

 

1. Introduction

Qualitative content analysis (QCA) has been defined as a method for systematically describing qualitative data by assigning successive parts of the text to the categories of a coding frame (SCHREIER, 2013). We highlight that QCA is based on systematic, rule-bound procedures, but cannot be reduced solely to a coding frame and remains a human-generated interpretation (grounded in hermeneutic tradition) (GLÄSER & LAUDEL, 2013; MAYRING, 2014). Generally, in clinical psychology, especially in the field of clinical assessment, there is a distinctive presence of QCA in case conceptualization for psychotherapy purposes and research tradition. Both inductive and deductive approaches (ELO & KYNGÄS, 2008; MAYRING, 2000) are used in systematic research and practical professional assessment. [1]

This definition of QCA reveals a version of content analysis that we would like to believe is a tool of scientific cognition that, on the one hand, allows us to conduct repetitive and intersubjective communicative empirical research, but on the other hand, also allows us to maintain contact with deep layers of meanings, both those that are intentionally conveyed by the subjects and those that may emerge in the process of active interpretation and understanding gained by the person who reads, listens or co-participates in the study. At the same time, in clinical psychology we—as researchers and practitioners—are dealing with research that is linked in different ways to QCA, but will meet the definition criteria mentioned above to varying degrees. For instance, there are many interpretive systems such as the social cognition and object relations scale (PETERS, HILSENROTH, EUDELL-SIMMONS, BLAGYS & HANDLER, 2007) for thematic apperception test (TAT, MURRAY, 1943) interpretation, and more comprehensive methods such as the core conflictual relationship theme (CCRT) developed by LUBORSKY and CRITS-CHRISTOPH (1998) that encompasses both tailor-made categories and standard categories. Even the dream analysis of Sigmund FREUD is mentioned in this area (MAYRING, 2014). However, these research and professional activities are not always identified as being representative of QCA, which is why we will present a brief review of such studies in the field of clinical psychology. At the same time it is worth mentioning that the subject of interest here is not quantitative content analysis in its classic, traditional form associated with reporting the frequency of words or various surface properties of the text, although nowadays it is also present in clinical psychology in the form of computer-aided lexical analyses (PENNEBAKER, FRANCIS & BOOTH, 2001). [2]

Clinical psychology is both a field of professional practice and a scientific discipline. As a psychological specialty practitioners provide ongoing and comprehensive mental and behavioral healthcare for individuals and families, consultation to agencies and communities, training, education and supervision, and research-based practice (AMERICAN PSYCHIATRIC ASSOCIATION, 2013). As a scientific discipline, practitioners of clinical psychology are devoted to description (of mental health and disorders) and explanation (of mechanisms for mental health and disorders based on psychological theory and empirical research) in the context of knowledge about biological, psychological and sociocultural factors (CIERPIAŁKOWSKA & SĘK, 2017). Furthermore, clinical psychology has been closely associated with psychotherapy both as a service and as an area of empirical research, but we will not concentrate on the differences between clinical psychology and psychotherapy here. [3]

In this article we reflect on how QCA as a research method is incorporated into clinical psychology as a scientific discipline. We start by showing the main points of convergence between QCA and clinical psychology and the diversity of QCA-informed studies encountered in the field of psychological disorders (Section 2). In the last part of the paper, we share a view on the challenges faced when using QCA in a more specific clinical context, namely, explanatory assessment of personality disorders (Section 3). We end with a summary of these reflections (Section 4). The reflections presented here appeared in connection with a summary of work in which we analyzed the statements of people with different levels of personality organization, where the main content was autobiographical stories structured as narratives (SOROKO, 2014). [4]

2. Qualitative Content Analysis in Clinical Psychology

2.1 The place of QCA in the field of clinical psychology

There are areas of potential dialectics immanent in clinical psychology (CIERPIAŁKOWSKA, GÓRSKA & SOROKO, 2018), namely science and practice, assessment and psychotherapy; and descriptive and explanatory assessment. In our opinion, QCA is important for establishing connections and complementing these areas in many ways. We will discuss these dialectics briefly below, because the basic directions of research undertaken today with the use of QCA in clinical psychology are determined by these three areas determine (Figure 1).



Figure 1: The place of qualitative content analysis in the field of clinical psychology [5]

Firstly, we would like to focus on QCA as a method that is close to patients' experiences, and thus becomes useful for understanding complex aspects of human nature. Using it allows us to deal with verbal material elicited in an interpersonal situation based on psychological contact and relation, hence the recognition and development of QCA in the field of clinical psychology may contribute to bringing together research and practice. We believe that conducting close-to-the-practice studies may increase the validity of QCA, making the findings more credible and trustworthy for clinicians so that it has a potentially high impact on the practice. Thus, when we use QCA we have the opportunity to increase external validity, as well as to respond to practical needs when answering difficult, complex and unexploited scientific questions (e.g., patients' responses to early psychosis: JUDGE, ESTROFF, PERKINS & PENN, 2008; reasons for dropping out of psychotherapy: KHAZAIE, REZAIE, SHAHDIPOUR & WEAVER, 2016; emergence of psychotic content during the psychotherapy process: LEONHARDT et al., 2018). [6]

Secondly, QCA is an important method at the interface between diagnosis and psychotherapy in at least two major goals of clinical assessment: case formulation, and monitoring and evaluation of treatment outcomes. Case formulation (conceptualization) is a bridge between assessment of the patient and their optimal psychological intervention. Tracy D. EELLS (2007) believes that: "A case formulation helps organize information about a person, particularly when that information contains contradictions or inconsistencies in behaviour, emotion, and thought content" (p.4). [7]

The clinical judgmental task is universal, but given the quantity and ambiguity of information presented, it can often be perceived as daunting and prone to cognitive errors (FALVEY, 2001). The usefulness of QCA for clinical judgment cannot be overestimated here; using it enables us to work on interview and clinical observation material in combination with systematic principles of organizing this material according to specific rules (usually supported by psychological theory), leading to case conceptualization that is reliable, detailed and also of high predictive power (LUBORSKY & BARRETT, 2007; RABINOVICH & KACEN, 2010; SINGER & BONALUME, 2010a, 2010b). Moreover, the usefulness of QCA in clinical assessment is not limited to conceptualization of the case, but also encompasses the monitoring and evaluation of treatment and process of change (ANGUS et al., 2017; BORITZ, BRYNTWICK, ANGUS, GREENBERG & CONSTANTINO, 2014; BRAGA, OLIVEIRA, RIBEIRO & GONÇALVES, 2016; GONÇALVES et al., 2017). Thirdly, in clinical psychology QCA is a firm pillar for the specific task of case conceptualization, that is, uncovering an explanatory mechanism for a given disorder. [8]

Thus, in case conceptualization, the clinician goes beyond a descriptive diagnosis and requires an explanation of the pathomechanism of the disorder. We will briefly distinguish between the descriptive and explanatory aspects of assessment (CIERPIAŁKOWSKA, 2007). The descriptive aspect of clinical assessment is mainly used for differential and nosological diagnosis performed according to a biomedical model, and the use of structured tools is required (structured interviews or self-report methods of high psychometric quality). In contrast, for the explanatory aspect the use specific psychological theories is required to create hypotheses about the functioning of a person based on an understanding of internal psychological processes, intrapsychic structures and their explanatory role for the pathomechanism of disorder. For the purposes of an explanatory assessment it is highly useful to enrich data with qualitative methods (interviews and observations), but to interpret them in a structured, systematic and unbiased way. In other words, in an explanatory assessment, verbal and non-verbal material is being processed to generate a unique understanding of the patient, with psychological terms that capture not only direct content (e.g., declarations) but also hidden meanings (e.g., avoided feelings that are not present openly) suggested and predicted by the theoretical framework used. [9]

It is worth pointing out that in clinical psychology, the explanation is strongly embedded in a theoretical background, such as a psychodynamic, humanistic or cognitive-behavioral perspective. With any of these approaches, clinicians have a conceptual language, allowing both mental and behavioral processes and structures to be integrated into theoretical models useful for understanding and interpreting patients' functioning. Illustrative terms for structures are mental representations, cognitive schemas, internal object relations, ego. For processes, terms include emotional processing, emotional dis-regulation, internal conflicts, and defense mechanisms. [10]

The exemplary method grounded in a psychodynamic approach suitable for assessing a central pattern of relating with others is the above mentioned CCRT (LUBORSKY & BARRETT 2007), which is based on an analysis of relational episodes (narratives about the course of the significant relationships). Relational episodes are divided into three aspects that are assessed for their psychological content: wishes and desires of the self, responses in relationships with others, and responses of the self. If needed, it might be combined with the relationship anecdotes paradigm (RAP) interview (LUBORSKY, 1998), a method specifically designed to increase the frequency of narratives by asking the interviewee to tell several relationship episodes. The categories for assessing wishes and responses are both tailor-made and standardized (established through previous research). The CCRT is a useful tool, which is used specifically for case conceptualization for psychotherapy and psychotherapy itself (e.g., by shaping interpretations). A broader description of the CCRT here serves to highlight the main features of a successful clinical method based on QCA: it is based on psychological theory and paradigmatic assumptions that aspire to address intrapsychic structures and processes, such as object relations, internal conflicts or transference issues; researchers are able to elicit data in a form that increases the likelihood of proper assessment with the method; it can be used qualitatively or in a mixed methods approach; and it is useful in the practice of formulating interventions and at the same time is strongly supported by research. [11]

To sum up, implementation of QCA procedures in clinical psychology contributes to the improvement of external validity, the chance to address questions that are both complex and inspired by practice, the reorganization of clinically relevant information and the systematization of case conceptualization. [12]

2.2 Methodical diversity of studies using QCA in clinical psychology

Although projects in which researchers use QCA-based procedures may not be common in clinical psychology, the necessity or usefulness of qualitative analyses is often emphasized, which shows the evident expectation directed towards QCA. The research projects to date are very heterogeneous. While discussing the aspects of methodological diversity, we highlight the need for an awareness of methodological choices and their consequences, and we combine them into three groups: basic assumptions, data elicitation and data analysis. [13]

First of all, there are studies that meet different ontological and epistemological assumptions of qualitative studies and to varying degrees. There are differences among them according to such dimensions as rejection of positivism, adherence to the postmodern sensibility concerned with capturing the individual's perspective, examination of the constraints of everyday life, positioning researchers as insiders to what is being studied, and dealing with emerging theory instead of hypothesis testing (HOWITT, 2010). The definite starting point in these studies is a qualitative data format (verbal and observational data), but the philosophical background and the method of analysis are very different, which is why there is still an issue differentiating QCA from content analytic-informed studies; however, this will not be elaborated in this article. [14]

The research is conducted for idiographic purposes (learning about unique properties of individuals, e.g., case studies and/or examination of the process of change in psychotherapy) as well as for nomothetical purposes (searching for general rules that can be generalized to a larger population). [15]

Although QCA involves the study of people's experiences in order to grasp their individual perspective, examination of these experiences (e.g., in the coding process) may already vary in terms of focus on description or explanation. At one end, there is research that remains at the level of descriptive phenomena (explicit surface meanings), and the researcher is not looking for anything beyond what a participant has said or what has been written. At the other end of the dimension, the researcher goes beyond the direct content of the data and starts to search for the underlying ideas, assumptions and conceptualizations. This means that the reader makes interpretations with intentional use of a theory irrespective of whether it is common sense, naive or an empirically tested psychological theory. This distinction is made between semantic and latent themes in BRAUN and CLARKE (2006). [16]

Secondly, in clinical psychology research, QCA is carried out on various types of materials derived from different data elicitation methods. The most common are:

Materials are mostly verbal, obtained during interaction with a researcher or from written tasks derived from both scientific research and natural sources (psychotherapeutic sessions, diagnostic protocols or personal records). Sometimes the subject of analysis is video material created in similar contexts to those above. [18]

Thirdly, we would like to reflect on the widespread use of coding (scoring, interpretative, rating) systems (manuals) in clinical psychology. Although the aforementioned terms reflect the different nuances of these systems, here, for the sake of simplicity, we treat them equally. Considering the seemingly paradoxical suggestion formulated by Philipp MAYRING (2014) that QCA is a mixed-methods approach, we would like to develop this observation further. Mixing of qualitative and quantitative methods can take place in at least two areas—category development and category application (Figure 2). Categories are formulated in different ways: on the basis of theory (deductive), data (inductive) (HSIEH & SHANNON, 2005), or in a combined way relying on feedback loops of recursive comparisons between data and theory (MAYRING, 2014).



Figure 2: Coding system: development and application [19]

Regardless of the method chosen, there is a final set of output categories in which the data is described as well as possible. Sometimes this is sufficient as the final product of QCA, but at other times the resulting categories can be considered as a coding frame and deductively applied to many text passages to receive an analysis of frequencies of categories. The coding frame contains not only a list of categories, but also a description of the rules for assigning them, resulting from previous practices. If the frame is applied in the scope of texts obtained within a specific research project, or the unified group of texts for which the frame is suitable is well defined, we are dealing with a "local" category application. The term "local" means here that the usefulness of the frame is constricted to a specific group of texts. It is possible then to specify the parameters of its validity and reliability, as well as make corrections in the coding frame itself, if necessary. Furthermore, the coding frame remains linked to the data from which it was created, which makes it more effective for the researcher to express essential meanings. If, on the other hand, the coding frame becomes an autonomous coding system and is used widely in nomothetic research ("global" category application), then its qualitative nature alters. The categories then can be treated simply as variables, equivalent to other quantitative variables (Figure 2). "Global" category application leads to the separation of a coding frame from a specific group of texts toward more independent measures of well-defined constructs. The definitions are justified by psychological theory, which becomes a significant point of reference here. [20]

Coding frames in clinical psychology often evolve towards coding systems (or manuals) (SINGER & BONALUME, 2010a; STEIN et al., 2012). These systems are characterized by precise coding rules, focusing on effective training of coders and emphasis on the highest possible compatibility of independent coding (e.g., inter-coder agreement). At the same time, this generates the risk of interference with the validity of the coding, because the new material may have some unchecked properties that were not taken into account during the system development stage, and that determine the content obtained (e.g., the frame was based on the TAT stories and is then reused with personal narrative material; then the autobiographical content in the latter is treated differently, not as a projection but mainly as a biographical memory; and then it may or may not be important in light of the particular research goal). [21]

Importantly, in research-reporting practices a detailed description of the coding system is recommended (referring to the idea of an unbiased experimenter whose impact on data should be minimized). Although as researchers we refer to human coding (not computerized), in which a certain degree of interpretation is expected and we make sure that coders are proficient, well-trained, etc., but we rarely reflect on which cognitive and emotional processes are evoked during this procedure. In psychological assessment there is an ongoing debate on clinical judgment and the determinants, consequences and preventive actions of this judgment, such as self-monitoring of one's own participation in the assessment process (GARB, 2010). [22]

There are precautions to be taken in order to prevent the "global" coding system from being too far removed from the data, such as careful examination of application conditions, reorganization of the coding system to adjust it to the conditions, and rethinking the theoretical assumptions underpinning the coding system (e.g., the CCRT is used according to the basic assumption of internal object relations, but is sometimes used as a system for assessing interpersonal—social relations, which is contrary to the original version and should be treated with caution: SOROKO & CIERPIAŁKOWSKA, 2018). [23]

Coding systems—in connection with practical problems in applying them—evolve. Conducting a literature analysis allows us to distinguish two strategies of the evolution. Firstly, researchers transform coding systems to achieve better validity by capturing as much meaningful data as possible. This is the case, for example, with the narrative emotional process coding system (NEPCS 2.0, ANGUS et al., 2017; BORITZ et al., 2014), in which the method for analysis of transcripts evolved into analysis of video. Similarly, the Innovative Moments coding system (GONÇALVES et al., 2017) was expanded toward a broader exploration amenable to processual aspects of psychotherapeutic change. Secondly, coding systems can be specialized by defining precisely, in terms of general psychology, the unit of analysis (as in autobiographical memory narrative: SINGER & BONALUME, 2010a, 2010b). In effect, these systems are limited to a well-defined but narrower area. Thus, coding systems in a process of repetitive global application of qualitatively developed categories become more and more autonomous and, although they are developed and adapted to meet the demands of scientific study, are most often an expression of envisaging and applying QCA in a technical way. The consequences of the transformation should be openly discussed. [24]

3. Qualitative Content Analysis in Clinical Assessment of Personality Disorders

3.1 The specificity of clinical assessment of personality disorders

After this general review, it is worth posing a more detailed question that, due to our clinical interests and research experience, is focused on an explanatory assessment (which is the basis for case conceptualization) in the area of personality disorders. We will briefly define personality disorders, then highlight our main areas of concern, and finally show the main problems and potential attempts to solve them. [25]

Personality disorders are long-term patterns of behavior and inner experience (thinking, feeling, perceiving) that deviate from the expectations of the culture. The pattern of experience and behavior begins in late adolescence or early adulthood, and causes distress or problems in functioning. The essential features of a personality disorder are impairments in personality (self and interpersonal) functioning and the presence of pathological personality traits (AMERICAN PSYCHIATRIC ASSOCIATION, 2013). A disturbed pattern of functioning is often not recognized by the person as her/his own problem (ego-syntonic) thus it is not prone to an adequate self-report, and the difficulties it provokes are rather seen by the person as coming from external sources, such as the character of other people. [26]

Currently, in the personality disorders field the discussion is primarily about two issues: a shift of thinking and research on personality disorders from the categorical model (specific types of disorders) to the dimensional model (dimensions of disturbed and/or healthy personality traits) (HOPWOOD, 2018; WAUGH et al., 2017), and inadequacy of claims of rigid, repetitive patterns of perceiving and learning about reality, its emotional experience, social relations and drive control, as relatively high variability in this type of disorder has been observed (SKODOL et al., 2005). Thus, by dealing here only with the latter, there is an urgent necessity to explain the situational variability of patients' functioning, knowing that the variability is triggered both externally (events) and internally (experiences). For example, in the field of research on mentalization capacity (ability to recognize mental states of self and others) it has been discovered that mentalization is more disturbed when a person is aroused by an emotional-relational stimulus (e.g., waiting for a late friend while being vulnerable to negligence due to early childhood experiences) than in an impersonal or/and non-significant situation (e.g., waiting for a late friend without such experiences versus feeling close to a friend despite difficult experiences from the past). The same person may mentalize differently depending on the configuration of stimuli—which may or may not activate internal structures—and his/her capacity to mentalize will fluctuate (GÓRSKA, 2018). [27]

The task of explaining the situational variability among people with personality disorders has already been undertaken from different theory-based therapeutic perspectives, where intra-psychic structures and processes are delineated, assessed and then become a target for therapeutic intervention. For instance, role schema modes in schema therapy (KELLOGG & YOUNG, 2006) or rules of internal dyad activation in psychodynamic object relation theory (KERNBERG, 2005). However, the explanation still remains a challenge, especially for scientific empirical studies and evidence-based models for personality disorder assessment (HOPWOOD, 2018). [28]

Among the main challenges for the explanatory assessment of personality disorders are the following (CLARKIN, MEEHAN & LENZENWEGER, 2015; McCLOUGH & CLARKIN, 2005; SOROKO & GÓRSKA, 2018):

3.2 The role of QCA in the explanatory assessment of personality disorders

While conducting QCA in the area of personality disorders, we face some problems that are specific to this area. Below are some particularly important ones, along with some promising ideas on how to deal with them in research practice (Figure 3). The problems correspond to the overall challenges in the field of personality disorder assessment mentioned above, and include: detecting clinically relevant information and explaining the pathomechanism, access to unconscious internal processes, and including relational information in the dataset.



Figure 3: QCA directed to explanation of pathomechanism in personality disorders [30]

3.2.1 Detecting clinically relevant information—balancing theory and data

The aim of our use of QCA here is to solve a problem of diagnostic nature, i.e., to answer the question of whether the presented material contains indicators of specific traits, states or mental structures and processes (e.g., "feeling of rejection," "splitting," "separation," "therapeutic change"). When considering the category generation phase, external knowledge (theory-driven approach) is used to a large extent. In consequence, the units of analysis are separated from the descriptive or phenomenological aspects of experiences, which could lead to an over-deductive approach that omits the actual content of the material. At the same time, from clinical experience we know, for example, that there are many different manifestations of these intrapsychic processes that are specific to the individual, their experience and the social and cultural context in which they live. Vigilance and openness to unexpected information are therefore essential to fully accomplish this diagnostic task. For example, in studies of psychotherapy change with the use of the ambivalence resolution coding system (BRAGA et al., 2016), it was noticed that markers of more integrative and less integrative ways of resolving ambivalence co-occurred, contrary to the expectation of a more integrative way at the end of treatment. Researchers therefore must have been open to the occurrence of unexpected categories in spite of theoretical assumptions. [31]

Thus, in developing coding frames for use by independent raters, the two-track creation of a set of categories by inductive analysis of data and by deductive analysis from theory is crucial, with as many feedback loops as needed (MAYRING, 2000). Jochen GLÄSER and Grit LAUDEL (2013) insist that the inductive correction of a deductively obtained set of categories weakens the role of theory and they warn of immediately abandoning theory whenever there is a conflict between data and theory. However, we have learned from our practice that it is possible to combine information derived from data and assumed by theory, in order to make the understanding broader, more saturated and better organized. [32]

3.2.2 In search of explanatory power—patterns in and between categories

With a view on the explanatory purpose of using QCA, the problem remains how to gain knowledge of the explanatory status by analyzing qualitative data. GLÄSER and LAUDEL asked this question about social problems, also emphasizing the necessity to integrate existing theory with patterns identified in the data, in order to obtain causal arguments and mechanistic explanations. They pointed out two requirements in the search for patterns in qualitative data: rich descriptions of empirical phenomena and variance in the data. [33]

However, as a basis for explanation, not only is recognizing patterns in the data recommended, but also the use of theory (BAZELEY, 2009) and a search for conceptual connections (RABINOVICH & KACEN, 2010). We have already partially considered the former, and the latter, as GLÄSER and LAUDEL (2013) state, is inevitably based on some degree of standardization of idiosyncratic descriptions in the original text. Thus, a procedure of category transformations is needed. RABINOVICH and KACEN (2010) prefer relations between categories analysis to guide this process. They report three key relationship types (contained, temporal and causal relationships) and supplement this with an additional dimension of comparison between categories: Bilateral, trilateral and quadrilateral. Although our aim is not to present this analysis in detail, it is of such importance in the context of this explanatory diagnosis that we will expand on it slightly here. Bilateral relationships essentially denote a pattern in which one category recurs often in conjunction with another. It is thus pertinent to ask the interpretive question: What meaning does one category acquire in the presence of another? We may then assume that it reflects the participants' conscious or unconscious association. In other words, these co-occurrences might suggest a rationale for conceptualization where we recognize the trigger (one category) of the behavior (another category) in a certain context. Trilateral relationships (when one category links to two others, maintaining the same relationship pattern with each) are helpful in recognizing, for example, relationship patterns when we have two apparently distinct stories about important others. Quadrilateral relationships (derived from trilateral relationships, but instead of having two categories connected by a third, they are connected by the substructure of two opposite or different categories) are very useful for recognizing the external or internal stimuli that direct behavior (situational variability) (RABINOVICH & KACEN, 2010, 2013). [34]

To sum up, the recognition of patterns of relationships between categories, especially if these patterns can be allied with theory, is a promising way to serve explanatory goals in both idiographic and nomothetic studies. [35]

3.2.3 Dealing with unconscious and evolving speech—momentary processual analysis

An analysis of relations between categories proved to be quite useful for encompassing such aspects of human functioning as unconscious intentions (e.g., verbally denied aggressive motives) or unintended meanings (e.g., slips of the tongue). However, there are other remaining problems stemming from interrelations between phenomenological descriptions of experiences made by disturbed patients. Sometimes the phenomena cannot be well described by patients because it is this inability that constitutes their psychopathology. Within the problem is an expectation that research participants are considered experts in the phenomenon studied, and that researchers aspire to investigate their perspective. Participants, however, supply both conscious and unconscious knowledge, parts of which may contradict others (RABINOVICH & KACEN, 2010) and contradict external knowledge, e.g., the psychopathology. [36]

An example is research and reflection on how patients with borderline personality disorder experience sadness in a relational context (BRIAND-MALENFANT, LECOURS & DESCHENAUX, 2012). The researchers analyzed the interviews conducted with patients and coded them using the interpretative phenomenological analysis principles with an inductive and exploratory approach. The themes they derived from the data indicated that sadness analyzed at the level of the phenomenon present in patients' narratives was not actually characterized by important defining properties of sadness. Unexpectedly, sadness was not associated with a representation of loss and there was an absence of themes of mourning. Perhaps instead of sadness in analyzed episodes, there is a non-mentalized experience that should not be conceptualized as proper sadness. The authors showed that this result did not appear to rest on an experimental artifact but—in comparison with non-borderline cases—on the subjective experience characteristic of borderline pathology. [37]

Hence, it is evident that as a result of conducting QCA, we do not receive what we assume to be the main subject of our research, but rather what can be obtained through a specific method of data collection, mediated by psychopathological processes. In this case, the experience of sadness elicited by the interview method appeared in an unprocessed, distorted version. Psychopathology, especially personality disorders, can be the subject of QCA, but can also have an influence on the obtained results (e.g., categories) in a way that distorts the analyzed content. [38]

Remarkably, communicative validation as a form of quality control is restricted in personality disorder research. Individuals with personality disorders experience their symptoms as ego-syntonic, so it is hard to communicate on this subject directly. Moreover, content analytic categories are often highly theoretical and it would be harmful to talk in psychological jargon about what are often tentative experiences. [39]

Some internal psychological processes, for example defensive mechanisms, are quite well recognized in verbal and non-verbal expression (PERRY & HENRY, 2004) but, once operating, change the availability or sharpness of other psychological processes in speech. We have encountered this issue when studying the defensive activity of borderline patients and their capacity for mentalization (JAŃCZAK, GÓRSKA & SOROKO, 2018). In this study we observed the influence of defensive activity on decreasing the detection of mentalization capacity. We believe that internal processes, especially the processing of emotional-relational experience (GÓRSKA & SOROKO, 2017), can be greatly enriched by sequencing speech into the specific stages of experience processing and performing QCA on each step of this process separately. In part of our research sequencing was missing, so the QCA results were over-generalized and did not take into account the phase-based nature of emotional processing. This is in line with the notion that indices of emotional tone and abstraction (computer-assisted analysis of verbatim psychotherapy transcripts was performed) were differently dispersed, along with the phase of the patient–therapist linguistic interactions (MERGENTHALER, 2008). This urge for a more detailed temporal analysis is already being addressed, for example on a minute-by-minute basis, in therapy session videotapes in the NEPCS 2.0 (ANGUS et al., 2017). In this case, therefore, what can be called momentary processual analysis seems to be a productive path in QCA for explanatory purposes in clinical psychology. Researchers focus their attention on the process of experiencing, reflected in the process of telling about one's own experience. [40]

3.2.4 Inclusion of relational context—interviewers and coders as sources of data

Discrepancies between verbal accounts of experience and other information, both from non-verbal channels of communication (sometimes transcribed and subjected to analysis) and the type of relation the person forms with an interviewer (very rarely analyzed), are another relevant issue, combining the previously explored problem of unconscious meanings within the interview. The research interview is undoubtedly an interpersonal situation with all its emerging complexities (FRANCESCHINI, 2003; GOODBODY & BURNS, 2011; KVALE, 2008). From the point of view of a qualitative researcher, this means that the interview contains rich verbal and non-verbal material and information resulting from the relation between researcher and participant (SOROKO, 2015). [41]

Firstly, the inclusion of non-verbal information to gain emotional information involves such analytical strategies as introducing additional notations for non-verbal activity into transcripts and using them as equivalent to verbal data in coding or working on the transcript and audio recording simultaneously. It is not only about simply broadening the spectrum of phenomena that can be studied, but also about the possibility of acquiring categories from non-verbal data. As RABINOVICH and KACEN (2013) showed: "The category 'fulfilling needs' is connected with the category 'love' in a bilateral relationship, and it appears that in some nonverbal manner, M. is telling the interviewer, 'I will give you a drink and you will like me'" (p.221). [42]

The diagnostic and research value of the relationship features might also be considered. Many clients with personality disorders use primitive defense mechanisms and they induce elements of their own internal world in the clinician, who in turn may experience strong and initially overwhelming feelings. When paying close attention to these feelings, the clinician can use this information to better understand the client and clarify a possible diagnosis (McCLOUGH & CLARKIN, 2005). Transferring these clinical experiences to scientific research, we could be interested in data that are derived from the personal experience (such as feelings and associations) of the interviewer. They may record and share information about the relationship with an interviewee, which thus enriches the basic verbal and non-verbal data. QCA of notes or recordings might be initiated; knowing that self-observation is limited, this data source can be used as a complementary one. Hence, the subjectivity of the researcher may become an instrument of cognition in a broader sense (HOLLWAY, 2009). [43]

Relatively little attention is paid to the coders' perspective and the process of organizing data and interpretation as a human-made process. In a qualitative approach, the researcher's reflexivity and socio-psychological background (HABERMAS & BLUCK, 2000) is highlighted, and in nomothetically oriented studies, the professionalism of coders and their training is evident. As we mentioned earlier, still less attention is paid to the psychological situation of the person working with the text, although this would be a very valuable topic. Researchers using QCA can draw inspiration from clinical practice not only at the stage of qualitative material generation (e.g., recognizing own emotional attitudes toward a participant), but also at the stage of qualitative analysis (e.g., monitoring own emotional and cognitive processes and being aware of cognitive errors and biases while analyzing data). [44]

While conducting research on the self-narrative activity of people with personality disorders (SOROKO, 2014) using a global category application with the help of trained coders, we noticed that their task has double significance. Tasks performed by independent coders are not only a valuable way to obtain agreement of coding, but also to reach information from the so-called 'level of text reception'—emphasizing reading, listening or experiencing the text. Thus, we may expect that the content of the impressions, feelings or associations reported by coders could contain information, especially about relational patterns, that might never have been discovered. [45]

Furthermore, the training of coders is a rule-guided process aiming at comparable results. As such it may restrain the possibility to discover less obvious interpretations. An additional or sometimes alternative method may be the use of expert (or non-expert) text readings to determine the range of responses it stimulates, showing a set of possible overlapping, independent or conflicting readings. In the course of the deliberations presented here, the following question resounds more and more clearly in the background, so we would like to emphasize it: Why, then, do we still consider coding frames to be qualitative and not quantitative? We believe that it is not because they are developed for qualitative material (mainly text), nor because they are derived from the analysis of meanings, quantified to a different degree. They can be understood as qualitative because of the human coding process, and therefore as interpretative, and in consequence far more constructive. [46]

4. Summary

In this article we have attempted to reflect on the methodical aspect of QCA in the field of clinical psychology, with particular emphasis on the task of explaining the pathomechanism of personality disorders. Practitioners of clinical psychology, with the use of QCA, are able to better fulfill their tasks, namely, to bridge research and practice and to integrate the assessment (especially case conceptualization) and psychotherapy. Moreover, hopefully we have given justification for the usefulness of QCA for explanatory assessment in the field of personality disorders, especially highlighting that conventional self-reporting may be inadequate for valid assessment of personality disorders (McCLOUGH & CLARKIN, 2005). [47]

On the other hand, from the perspective of using QCA as a qualitative method in the field of personality disorders, challenges could be depicted around three issues that perhaps are not limited to QCA in personality disorder assessment:

We would recommend the use of QCA to researchers interested in mental disorders, with particular encouragement to take into account the human processes occurring during interview and qualitative analysis. In our opinion, these phenomena should be reflected, recorded, documented and analyzed as a source of important information aimed at understanding the psychological phenomena under study. Although, of course, not all the reflections presented here can be easily agreed upon, we hope that many of them will seem familiar to the readers and are worthy of further discussion. [49]

References

American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: Author.

Angus, Lynne E.; Boritz, Tali; Bryntwick, Emily; Carpenter, Naomi; Macaulay, Christianne & Khattra, Jasmine (2017). The narrative-emotion process coding system 2.0: A multi-methodological approach to identifying and assessing narrative-emotion process markers in psychotherapy. Psychotherapy Research, 27(3), 253-269.

Bazeley, Pat (2009). Analysing qualitative data: More than "identifying themes". Malaysian Journal of Qualitative Research, 2(2), 6-22.

Boritz, Tali Z.; Bryntwick, Emily; Angus, Lynne; Greenberg, Leslie S. & Constantino, Michael J. (2014). Narrative and emotion process in psychotherapy: An empirical test of the narrative-emotion process coding system (NEPCS). Psychotherapy Research, 24(5), 594-607.

Braga, Catia; Oliveira; Joao T.; Ribeiro, Antonio P. & Gonçalves, Miguel M. (2016). Ambivalence resolution in emotion-focused therapy: The successful case of Sarah. Psychotherapy Research, 28(3), 423-432.

Braun, Virginia & Clarke, Victoria (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101.

Briand-Malenfant, Rachel; Lecours, Serge & Deschenaux, Emilie (2012). What does sadness mean to BPD patients?. Journal of Personality Disorders, 26(6), 939-955.

Cierpiałkowska, Lidia (2007). Psychopatologia [Psychopathology]. Warszawa: Wydawnictwo Naukowe Scholar.

Cierpiałkowska, Lidia & Sęk, Helena (2017). Scientific and social challenges for clinical psychology. Roczniki Psychologiczne/Annals of Psychology, 19(3), 419-436.

Cierpiałkowska, Lidia; Górska, Dominika & Soroko, Emilia (2018). Dialectics of professional development of clinical psychologist. Plenary lecture, 3rd National Conference of Clinical Psychology, November 22-24, Poznań, Poland.

Clarkin, John F.; Meehan, Kevin B. & Lenzenweger, Mark F. (2015). Emerging approaches to the conceptualization and treatment of personality disorder. Canadian Psychology/Psychologie Canadienne, 56(2) 155-165.

Cramer, Phebe & Blatt, Sidney J. (1990). Use of the TAT to measure change in defense mechanisms following intensive psychotherapy. Journal of Personality Assessment, 54(1-2), 236-251.

Eells, Tracy D. (2007). History and current status of psychotherapy case formulation. In Tracy D. Eells (Ed.), Handbook of psychotherapy case formulation (pp.3-32). London: Guilford Press.

Elo, Satu & Kyngäs, Helvi (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107-115.

Falvey, Janet E. (2001). Clinical judgment in case conceptualization and treatment planning across mental health disciplines. Journal of Counseling & Development, 79(3), 292-303.

Franceschini, Rita (2003). Unfocussed language acquisition? The presentation of linguistic situations in biographical narration. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 4(3), Art. 19, http://dx.doi.org/10.17169/fqs-4.3.667 [Date of Access: December 4, 2019].

Garb, Howard N. (2010). The social psychology of clinical judgment. In James E. Maddux, June Price Tangney (Eds.), Social psychological foundations of clinical psychology (pp.297-311). New York, NY: Guilford Press.

Gläser, Jochen & Laudel, Grit (2013). Life with and without coding: Two methods for early-stage data analysis in qualitative research aiming at causal explanations. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 14(2), Art. 5, http://dx.doi.org/10.17169/fqs-14.2.1886 [Date of Access: August 28, 2019].

Gonçalves, Miguel M.; Ribeiro, António P.; Mendes, Inês; Alves, Daniela; Silva, Joana; Rosa, Catarina; Braga, Cátia; Batista, João; Fernández-Navarro, Pablo & Oliveira, João T. (2017). Three narrative-based coding systems: Innovative moments, ambivalence and ambivalence resolution. Psychotherapy Research, 27(3), 270-282.

Goodbody, Louise & Burns, Jan (2011). A disquisition on pluralism in qualitative methods: The troublesome case of a critical narrative analysis. Qualitative Research in Psychology, 8(2), 170-196.

Górska, Dominika (2018). Fluctuations of mentalization in the context of relational stimuli and representational contents. Current Issues in Personality Psychology, 6(4), 279-291.

Górska, Dominika & Soroko, Emilia (2017). Between verbalization and reflection: Studies on referential activity and narrative processes in borderline personality organization. Psychoanalytic Psychology, 34(4), 422-433.

Habermas, Tilmann & Bluck, Susan (2000). Getting a life: The emergence of the life story in adolescence. Psychological Bulletin, 126(5), 748-769.

Hollway, Wendy (2009). Applying the "experience-near" principle to research: Psychoanalytically informed methods. Journal of Social Work Practice, 23(4), 461-474.

Hopwood, Christopher J. (2018). Interpersonal dynamics in personality and personality disorders. European Journal of Personality, 32(5), 499-524.

Howitt, Dennis (2010). Introduction to qualitative methods in psychology. London: Prentice Hall.

Hsieh, Hsiu-Fang & Shannon, Sarah E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277-1288.

Jańczak, Monika; Górska, Dominika & Soroko, Emilia (2018). Procesy obronne i mentalizowanie w opowieściach o relacjach u pacjentów borderline. – badania ilościowe i jakościowe [Defensiveness and mentalization in stories about close relationships in borderline patients. Quantitative and qualitative study]. Presentation at the "III Krajowa Konferencja Psychologii Klinicznej", Institute of Psychology, Adam Mickiewicz University, Poznań, Poland, November 22-24, 2018.

Judge, Abigail M.; Estroff, Sue E.; Perkins, Diana O. & Penn, David L. (2008). Recognizing and responding to early psychosis: A qualitative analysis of individual narratives. Psychiatric Services, 59(1), 96-99.

Kellogg, Scott H. & Young, Jeffrey E. (2006). Schema therapy for borderline personality disorder. Journal of Clinical Psychology, 62(4), 445-458.

Kernberg, Otto F. (2005). Object relations theories and technique. In Ethel S. Person, Arnold M. Cooper, Glenn O. Gabbard (Eds.), Textbook of psychoanalysis (pp.57-75). Washington, D.C.: American Psychiatric Publishing.

Khazaie, Habibolah; Rezaie, Leeba; Shahdipour, Niloofar & Weaver, Patrick (2016). Exploration of the reasons for dropping out of psychotherapy: A qualitative study. Evaluation and Program Planning, 56, 23-30.

Kuncewicz, Dariusz & Kuncewicz, Dorota (2019). Hidden stories in monologues. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 20(1), Art. 9, http://dx.doi.org/10.17169/fqs-20.1.3005 [Date of Access: December 4, 2019].

Kuncewicz, Dariusz; Kuncewicz, Dorota; Sokołowska, Ewa & Sobkowicz, Jolanta (2015). What are you saying when you are talking about ...? Procedure for isolating a hidden story in a monologue about the author's own life. Polish Journal of Applied Psychology, 13(2), 25-46.

Kvale, Steinar (2008). Doing interviews. London: Sage.

Leonhardt, Bethany L.; Kukla, Marina; Belanger, Elizabeth; Chaudoin-Patzoldt, Kelly A.; Buck, Kelly D.; Minor, Kyle S.; Vohs, Jenifer L.; Hamm, Jay A. & Lysaker, Paul (2018). Emergence of psychotic content in psychotherapy: An exploratory qualitative analysis of content, process, and therapist variables in a single case study. Psychotherapy Research, 28(2), 264-280.

Luborsky, Lester (1998). The relationship anecdotes paradigm (RAP) interview as a versatile source of narratives. In Lester Luborsky & Paul Crits-Christoph, Understanding transference: The core conflictual relationship theme method (2nd ed., pp.109-120). Washington, D.C.: American Psychological Association.

Luborsky, Lester & Barrett, Marna S. (2007). The core conflictual relationship theme: A basic case formulation method. In Tracy D. Eells (Ed.), Handbook of psychotherapy case formulations (pp.105-135). London: Guilford Press.

Luborsky, Lester & Crits-Christoph, Paul (1998). Understanding transference: The core conflictual relationship theme method. New York, NY: American Psychological Association.

Lysaker, Paul. H.; Ringer, Jamie; Maxwell, Christina; McGuire, Alan & Lecomte, Tania (2010). Personal narratives and recovery from schizophrenia. Schizophrenia Research, 121(1-3), 271-276.

Marszał, Monika (2015). Mentalizacja w kontekście przywiązania: zdolność do rozumienia siebie i innych u osób z osobowością borderline [Mentalization in the context of attachment: Ability to understand oneself and others in people with borderline personality]. Warszawa: Difin.

Mayring, Philipp (2000). Qualitative content analysis. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 1(2), Art. 20, http://dx.doi.org/10.17169/fqs-1.2.1089 [Date of Access: August 28, 2019].

Mayring, Philipp (2014). Qualitative content analysis: Theoretical foundation, basic procedures and software solution, http://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173 [Date of Access: August 28, 2019].

McClough, Joel F. & Clarkin John F. (2005). Personality disorders. In Michel Hersen (Ed.), Psychological assessment in clinical practice: A pragmatic guide (pp.117-146). London: Routledge.

Meehan, Kevin B.; Cain, Nicole M.; Clarkin, John F. & De Panfilis, Chiara (2018). The impact of integrated object representations on rejection sensitivity. Psychoanalytic Psychology, 35(2), 254-259.

Mergenthaler, Erhard (2008). Resonating minds: A school-independent theoretical conception and its empirical application to psychotherapeutic processes. Psychotherapy Research, 18(2), 109-126.

Murray, Henry Alexander (1943). Thematic apperception test. Cambridge, MA: Harvard University Press.

Pennebaker, James W.; Francis, Martha E. & Booth, Roger J. (2001). Linguistic inquiry and word count: LIWC 2001. Mahway, NJ: Lawrence Erlbaum Associates.

Perry, J. Christopher & Henry, Melissa (2004). Studying defense mechanisms in psychotherapy using the defense mechanism rating scale. In Uwe Hentschel, Dudmund Smith, Juris G. Draguns, Wolfram Ehlers (Eds.), Defense mechanisms: Theoretical, research and clinical perspectives (pp.165-192). Amsterdam: Elsevier.

Peters, Eric J.; Hilsenroth, Mark J.; Eudell-Simmons, Erin M.; Blagys, Matthew D. & Handler, Leonard (2007). Reliability and validity of the social cognition and object relations scale in clinical use. Psychotherapy Research, 16(5), 617-626.

Pistole, Debbie R. & Ornduff, Sidney R. (1994). TAT assessment of sexually abused girls: An analysis of manifest content. Journal of Personality Assessment, 63(2), 211-222.

Rabinovich, Merav & Kacen, Lea (2010). Advanced relationships between categories analysis as a qualitative research tool. Journal of Clinical Psychology, 66(7), 698-708.

Rabinovich, Merav & Kacen, Lea (2013). Qualitative coding methodology for interpersonal study. Psychoanalytic Psychology, 30(2), 210-233.

Schreier, Margrit (2013). Qualitative content analysis. In Uwe Flick (Ed.), The Sage handbook of qualitative data analysis (pp.170-183). London: Sage.

Singer, Jefferson A. & Bonalume, Laura (2010a). Autobiographical memory narratives in psychotherapy: A coding system applied to the case of Cynthia. Pragmatic Case Studies in Psychotherapy, 6(3), 134-188.

Singer, Jefferson A. & Bonalume, Laura (2010b). Toward the scientific study of autobiographical memory narratives in psychotherapy. Pragmatic Case Studies in Psychotherapy, 6(3), 215-222.

Skodol, Andrew E.; Gunderson, John G.; Shea, M. Tracie; McGlashan, Thomas H.; Morey, Leslie C.; Sanislow, Charles A.; Bender, Donna S.; Grilo, Carlos M.; Zanarini, Mary C.; Yen, Shirley; Pagano, Maria E. & Stout, Robert L. (2005). The collaborative longitudinal personality disorders study (CLPS): Overview and implications. Journal of Personality Disorders, 19(5), 487-504.

Soroko, Emilia (2014). Aktywność autonarracyjna osób z różnym poziomem organizacji osobowości. Opowieści o bliskich związkach [Self-narrative activity of people with different levels of personality organization. Stories about close relationships]. Poznań: Wydawnictwo Naukowe UAM.

Soroko, Emilia (2015). Wkład psychoanalitycznego myślenia do rozwoju rozmowy psychologicznej jako jakościowej metody badawczej i diagnostycznej [Contribution of psychoanalytic thought on development of interview as a research and diagnostic method]. Polskie Forum Psychologiczne, 20(4), 514-535.

Soroko, Emilia & Cierpiałkowska, Lidia (2018). Levels of personality organization and internal relational patterns. Current Issues in Personality Psychology, 6(4), 292-304.

Soroko, Emilia & Górska, Dominika (2018). Beyond descriptions: Selected aspects of science and practice in clinical psychology and personality psychopathology. Current Issues in Personality Psychology, 6(4), 261-265.

Stein, Michelle B.; Slavin-Mulford, Jenelle; Sinclair, S. Justin; Siefert, Caleb J. & Blais, Mark A. (2012). Exploring the construct validity of the social cognition and object relations scale in a clinical sample. Journal of Personality Assessment, 94(5), 533-540.

Stein, Michelle B.; Slavin-Mulford, Jenelle; Siefert, Caleb J.; Sinclair, S. Justin; Smith, Michaela; Chung, Wei-Jean; Liebman, Rachel & Blais, Mark A. (2015). External validity of SCORS-G ratings of thematic apperception test narratives in a sample of outpatients and inpatients. Rorschachiana, 36, 58-81.

Waugh, Mark H.; Hopwood, Christopher J.; Krueger, Robert F.; Morey, Leslie C.; Pincus, Aaron L. & Wright, Aidan G. (2017). Psychological assessment with the DSM-5 alternative model for personality disorders: Tradition and innovation. Professional Psychology: Research and Practice, 48(2), 79-88.

Authors

Emilia SOROKO is an associate professor at the Department of Psychology and Cognitive Science, Adam Mickiewicz University, Head of the Qualitative and Mixed Methods in Clinical Psychology Research Lab. Her methodological interests involve qualitative approaches, especially interview methods, content analysis, narrative analysis, and mixed methods. She is involved mostly in research projects in clinical psychology (personality pathology assessment) and narrative psychology.

Contact:

Emilia Soroko, PhD

Department of Psychology and Cognitive Science, Adam Mickiewicz University
Szamarzewskiego 89
60-578 Poznań, Poland

Tel.: + 48 600936941
Fax: + 48 618292307

E-mail: soroko@amu.edu.pl
URL: soroko.home.amu.edu.pl

 

Michał DOLCZEWSKI is a PhD student at the Department of Psychology and Cognitive Science, Adam Mickiewicz University. His main research interests include clinical psychology, personality assessment, romantic relationships and a vast range of qualitative methods.

Contact:

Michał Dolczewski, MA

Department of Psychology and Cognitive Science, Adam Mickiewicz University
Szamarzewskiego 89
60-578 Poznań, Poland

Tel.: + 48 618292307

E-mail: micdol@amu.edu.pl

Citation

Soroko, Emilia & Dolczewski, Michał (2020). Qualitative Content Analysis in Clinical Psychology to Explain the Pathomechanism of Personality Disorders [49 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 21(1), Art. 19, http://dx.doi.org/10.17169/fqs-21.1.3432.

Forum Qualitative Sozialforschung / Forum: Qualitative Social Research (FQS)

ISSN 1438-5627

Creative Common License

Creative Commons Attribution 4.0 International License