Home > Public Resources > Trauma Blog > 2021 - April > Assessment of Reliability and Validity of a German Diagnostic Interview Module for ICD-11 Adjustment

Assessment of Reliability and Validity of a German Diagnostic Interview Module for ICD-11 Adjustment Disorder

Axel Perkonigg, Katja Beesdo-Baum, and Adreas Maercker

April 9, 2021

Adjustment disorder (AjD) is a relatively frequent mental disorder in various clinical settings and high-risk groups of the general population (Perkonigg et al., 2018). The clinical utility of the AjD diagnosis has been often criticized (e.g., Casey, 2014) and the disorder was under-researched for example due to the fact that few descriptive and analytical epidemiological data were available. In the past ten years, research on AjD increased partially due to more interest in stress related mental disorders but also due to the forthcoming fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association [APA], 2013) and to the 11th revision of the International Classification of Diseases (ICD-11; World Health Organization [WHO], 2018). In ICD-11, AjD has been conceptualized as a maladaptive reaction to one or multiple identifiable psychosocial stressors and was classified as “disorder specifically associated with stress” in the respective new group of the sixth ICD-11 chapter together with posttraumatic stress disorder (PTSD), Complex PTSD, prolonged grief disorder, as well as reactive attachment disorder and as disinhibited social engagement disorder, the latter both related to severe childhood distress. Symptoms of AjD are now characterized by two core symptom patterns in ICD-11: preoccupation with the stressor or its consequences and failure to adapt to it. Preoccupation might occur as excessive worry, recurrent and distressing thoughts, or constant rumination. Failure to adapt could for example be associated with sleep problems, concentration problems, or difficulties at work or with other duties. Significant impairment in various areas of functioning due to the symptoms is now mandatory for a diagnosis of ICD-11 AjD and it is expected that symptoms emerge within a month of the stressor and typically resolve within six months unless the stressor persists longer. Notably, the core symptoms of ICD-11 AjD are now conceptualized unique to AjD in contrast to the earlier ICD versions and in contrast to DSM-5. However, an ICD-11 AjD diagnosis is excluded if the symptoms are better explained by another mental disorder (i.e., other disorders specifically associated with stress or recurrent or single depressive episodes; see also ICD-11, 2018). In DSM-5, AjD is also constituted by a maladaptive adjustment in response to stress. However, the concept outlines the development of specified symptoms of depression, anxiety, and disturbances of conduct, forming six subtypes as for example AjD with depressed mood, or AjD with anxiety.

There are presently two screening instruments for ICD-11 AjD available: The Adjustment Disorder–New Module (ADNM) questionnaire (Maercker et al., 2007) and the recently published International Adjustment Disorder Questionnaire (IADQ; Shevlin et al., 2020). Because there was no diagnostic instrument available for ICD-11 AjD at the outset of our AjD-project we designed a new German standardized, clinical diagnostic interview module according to the actual definition and diagnostic guidelines of ICD-11 AjD (WHO, 2018; WHO GCP Network, 2019), that in addition allows to determine a DSM-5 AjD diagnosis (Perkonigg et al., 2015). In respect of the DSM-5 items the interrater reliability of the new module has been described as moderate for the AjD diagnosis (Hoyer et. al. 2020). For the present study we investigated interrater reliability, internal consistency, and factorial validity of the specific items that are used for its diagnostic algorithm and of the ICD-11 AjD diagnosis. In addition, we examined whether the interview module allows classifying respondents with ICD-11 AjD validly? 
Data from two studies were used. First, we used data from the DIA-X-5 test–retest study (Hoyer et al., 2020) to investigate interrater reliability of our new module by calculating various agreement indicators (i.e., Kappa). The DIA-X-5 is an updated DSM-5 version of the DIA-X/M-CIDI, a modularized, standardized, and computerized, clinical, diagnostic interview on major mental disorders (DSM-IV/ICD-10), originally based on the WHO Composite International Diagnostic interview. The new AjD module was computerized and added to the DIA-X-5 sections. The sample of the test-retest study for the DIA-X-5 consisted of 60 participants (age range: 15–67 years, M = 26.6 years), who were interviewed twice by different interviewers. The interviewers who conducted the retest assessment were blind regarding the results of the first interview. 

Secondly, we used data from the Zurich Adjustment Disorder Study (ZADS, N = 330) to examine internal consistency and construct validity by computing Cronbach’s alpha values and the Kuder–Richardson correlation coefficient along with confirmatory factor analysis (CFA) and latent class analysis (LCA). ZADS is a longitudinal study that included a baseline assessment and 6- and 12-month follow-up assessments. It was designed to investigate the prevalence, risk factors, course, and outcome of ICD-11 AjD in a high-risk sample of persons who had lost their jobs involuntarily (e.g., Perkonigg et al., 2018). In ZADS, the new AjD module was used as computerized and personal, clinical interview module with the DSM-IV version of the DIA-X/M-CIDI (Wittchen & Pfister, 1997) because the DIA-X-5 had not yet been available at the outset of the baseline assessment of ZADS. The module starts with a question on stressors experienced during 12 months prior to the interview, which is supplemented by an open list, describing stressors that can occur in everyone’s life. If one or more stressors are endorsed, their onset, frequency, and duration are assessed. At the end of this part, respondents are asked to identify their most distressing stressor to use as a reference for further inquiry if multiple stressors have been reported. An open description of the singular stressor or most distressing stressor is then requested. In the second part of the module, AjD symptoms are assessed, with the respondent using their singular or most severe stressor as a reference. In a third step, information about symptom onset, recency, and clinical significance is collected. Finally, items related to functional impairment regarding work, the household, school, leisure activities, and social contacts complete the module. The AjD CIDI module has been also added to the updated DIA-X-5 with few changes from the original version. Additional interviewer checks and confirmation were included to ensure that events had not already been reported in the preceding posttraumatic stress disorder (PTSD) section and that symptoms had been accordingly assessed.

In respect of the interrater reliability that was investigated in the test-retest study of the DIA-X-5, we found an adjusted kappa of 0.807 for the ICD-11 AjD diagnosis (Perkonigg et al, 2021). However, some specific items from the impairment and clinical significance-criterion performed poorly in the interrater study. This was mostly due to some items that are related to the “worst” period which the respondent had experienced with regard to the symptoms. We hypothesized that recall of a specific period during the course of the disorders may not have been as reliably rated as items for which respondents refer to the entire period after the stressor occurrence or onset.
Internal consistency was investigated with ZADS data. With Cronbach’s Alphas between .43 and .80, the assessment was still in an acceptable range. Results of a confirmatory factor analysis that investigated factorial validity yielded a superior fit for a one-factor model compared to a two-factor model. As we discussed in the related article, findings from some other studies had supported a bifactorial structure. However, some of the authors outlined that a unidimensional solution could also be taken into account as in some of the bifactorial findings, the factors were highly correlated.

Construct validity was investigated by latent class analysis of the ZADS data and an examination of factors associated with membership in the resulting classes. Supported by the fit indices, and closer inspection of competing models with regard to the probabilities of the symptom items, we favored a two-class model. Class 2 of this model included 94.3% of all ICD-11 AjD cases whereas member of class 1 yielded fewer or no symptoms and less or no impairment. In support of the construct validity of the ICD-11 AjD assessment we also showed that belonging to the ICD-11 AjD class 2 was associated with subjectively rated distress (OR = 2.18, 95% CI [1.57, 3.02]), and an external measure of the Brief Symptom Inventory (Franke et al., 2017), the global severity index (OR = 2.18, 95% CI [1.57, 3.02]. 
Despite the promising results, some limitations were outlined. For example, we discussed the relatively small number of participants in the interrater study and the particular subpopulation of individuals who had experienced job loss in the validity study. Due to the limitations of the two studies, it was outlined that further studies should demonstrate whether the promising first results for the new AjD module can be generalized to high-risk groups other than individuals who have experienced involuntary job loss. 

Reference Article

Perkonigg, A., Strehle, J., Beesdo‐Baum, K., Lorenz, L., Hoyer, J., Venz, J., & Maercker, A. (2021). Reliability and Validity of a German Standardized Diagnostic Interview Module for ICD‐11 Adjustment Disorder. Journal of Traumatic Stress

Discussion Questions

  • What are the differences between ICD-11 AjD and DSM-5 AjD? 
  • Why are diagnostic interviews of prime importance in assessing mental disorders?
  • What are the limitations of the two studies?

About the Authors

Axel Perkonigg, PhD, is a clinical psychologist and was a senior scientist at the Psychopathology and Clinical Intervention Unit of the University of Zurich. His research focus particularly comprises specific stress related disorders (i.e., PTSD and adjustment disorder), and substance use disorders in respect of their development, epidemiology and diagnostics. He is currently a lecturer for clinical psychology and addictions at the Baden-Württemberg Cooperative State University Stuttgart.
Katja Beesdo-Baum, PhD, is chair and full professor of Behavioral Epidemiology at the Institute of Clinical Psychology and Psychotherapy, Technische Universität Dresden (Germany). Her research focuses on the clinical descriptive and causal-analytic epidemiology of mental and behavioral disorders, the etiologic and pathogenic mechanisms of anxiety and depressive disorders, and the development of novel strategies for early identification and targeted intervention. She has been advisor to the Anxiety Disorder work group of the American Psychiatric Association Task Force for DSM-5 (2008-2012) and DSM-5-TR (since 2019).
Andreas Maercker, PhD, MD, is chair and full professor of Psychopathology and Clinical Intervention at the University of Zurich. He is co-director of the Institute’s outpatient clinic services. From 2011-2018 he chaired a work group at WHO for revising the International Classification of Diseases in the area of trauma- and stress-related disorders. In 2017, he was awarded the Wolter de Loos Award for Distinguished Contribution to Psychotraumatology in Europe from the European Society for Traumatic Stress Studies for his scientific work.

References Cited

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (fifth ed.).Washington, DC: American Psychiatric Association.

Casey, P. (2014). Adjustment disorder: New developments. Current Psychiatry Reports16, 451.

Franke, G.H., Jaeger, S., Glaesmer, H., Barkmann, C., Petrowski, K., & Braehler, E. (2017). Psychometric analysis of the brief symptom inventory 18 (BSI-18) in a representative German sample. BMC Medical Research Methodology 17, 14.

Hoyer, J., Voss, C., Strehle, J., Venz, J., Pieper, L., Wittchen, H.-U., Ehrlich, S., & Beesdo-Baum, K. (2020). Test–retest reliability of the Computer-Assisted DIA-X-5 Interview for Mental Disorders. BMC Psychiatry20(1), 280.

Perkonigg, A., Lorenz, L., & Maercker (2018). Prevalence and correlates of ICD-11 adjustment disorder: Findings from the Zurich Adjustment Disorder Study. International Journal of Clinical and Health Psychology18(3), 209–217.

Perkonigg, A., Strehle, J., Lorenz, L., Beesdo-Baum, K., & Maercker, A. (2015). Das AjD-CIDI Modul–standardisierte, klinische Diagnostik der Anpassungsstörung nach ICD-11 und DSM-5 [The AjD-CIDI module – standardized clinical diagnostics of the adjustment disorder according to ICD-11 and DSM-5]. Unpublished manuscript. University of Zurich and Technische Universitaet Dresden.

Shevlin, M., Hyland, P., Ben-Ezra, M., Karatzias, T., Cloitre, M., Vallières, F., Bachem, A., & Maercker A. (2020). Measuring ICD-11 adjustment disorder: The development and initial validation of the International Adjustment Disorder Questionnaire. Acta Psychiatrica Scandinavica141(3), 265–274.

Wittchen, H.-U., & Pfister, H. (1997). DIA-X-Interviews: Manual für Screening-Verfahren und Interview; Interviewheft Längsschnittuntersuchung (DIA-X Lifetime); Ergänzungsheft (DIA-X Lifetime); Interviewheft Querschnittuntersuchung (DIA-X 12 Monate); Ergänzungsheft (DIA-X 12 Monate); PC-Programm zur Durchführung des Interviews (Längs- und Querschnittuntersuchung); Auswertungsprogramm [DIA-X-interviews: manual for screening instruments and interview; interview booklet longitudinal assessment (DIA-X Lifetime); supplemental booklet (DIA-X Lifetime); interview booklet cross-sectional assessment (DIA-X 12 months); supplemental booklet (DIA-X 12 months); computer program for the interview (longitudinal- and cross-sectional assessment); analysis program]. Swets and Zeitlinger.

World Health Organization. (2018). International classification of diseases for mortality and morbidity statistics (11th Revision). Retrieved from

World Health Organization’s Global World Health Organization’s Global Clinical Practice Network. (2019, September 19). ICD-11 Guidelines. Adjustment disorder (with registration).