Elsevier

Drug and Alcohol Dependence

Volume 158, 1 January 2016, Pages 86-93
Drug and Alcohol Dependence

Full length article
Reproducibility and differential item functioning of the alcohol dependence syndrome construct across four alcohol treatment studies: An integrative data analysis

https://doi.org/10.1016/j.drugalcdep.2015.11.001Get rights and content

Highlights

  • Dimensionality of the alcohol dependence syndrome is reproducible across studies.

  • Differential item functioning was found for all measures of dependence severity.

  • Gender, age, marital status, and study predicted differential item responses.

  • Certain items are more reflective of dependence in some subgroups.

Abstract

Background

The validity of the alcohol dependence syndrome has been supported. The question of whether different measures of the construct are comparable across studies and patient subgroups has not been examined. This study examined the alcohol dependence construct across four diverse large-scale treatment samples using integrative data analysis (IDA).

Method

We utilized existing data (n = 4393) from the COMBINE Study, Project MATCH, the Relapse Replication and Extension Project (RREP), and the United Kingdom Alcohol Treatment Trial (UKATT). We focused on four measures of alcohol dependence: the Alcohol Dependence Scale (COMBINE and RREP), Alcohol Use Inventory (MATCH), the Leeds Dependence Questionnaire (UKATT), and the Diagnostic and Statistical Manual of Mental Disorders (COMBINE and MATCH). Moderated nonlinear factor analysis was used to create a measure of alcohol dependence severity that was moderated by study membership, gender, age, and marital status.

Results

A commensurate measure of alcohol dependence severity was successfully created using 20 items available in four studies. We identified differential item functioning by study membership, age, gender, and/or marital status for 12 of the 20 items, indicating specific patient subgroups who responded differently to items based on their underlying dependence severity.

Conclusions

Alcohol dependence severity is a single unidimensional construct that is comparable across studies. The use of IDA provided a strong test of the validity of the alcohol dependence syndrome and clues as to how some items used to measure dependence severity may be more or less central to the construct for some patients.

Introduction

The alcohol dependence construct is central to theories of problem alcohol use (Stockwell, 2015). However, the reproducibility of the alcohol dependence construct across studies and settings is not well known. There have been numerous psychometric studies of individual measures of alcohol dependence (Doyle and Donovan, 2009, Kahler et al., 2003a), but no studies have attempted to examine the construct of alcohol dependence across measures, studies, and patient subgroups. The goal of this study was to use integrative data analysis to test the construct of alcohol dependence severity among 4393 patients presenting for alcohol use disorder (AUD) treatment.

Edwards and Gross (1976) hypothesized that alcohol dependence syndrome was a collection of symptoms including narrowing of the drinking repertoire, salience of drink-seeking behavior, tolerance, drinking to relieve withdrawal symptoms, awareness of compulsion to drink, and return to drinking following abstinence. The dependence syndrome is primarily focused on physiological, psychological and behavioral symptoms that result in a core “drive to consume alcohol” (p. 79; Stockwell et al., 1979) and is therefore distinct from alcohol consumption, in and of itself, or negative consequences that result from alcohol use (Li et al., 2007). Almost 40 years of research on the dependence syndrome (Marshall, 2015, Stockwell, 2015) has supported the importance of the concept for the clinical formulation of AUD. Yet, the degree to which certain elements of the syndrome are central to the construct or subgroups (e.g., females) has received little attention (Kahler et al., 2003b), and no study has examined whether different measures of dependence are measuring the same construct across AUD samples.

IDA is a framework for merging, analyzing, and comparing information from multiple studies (Curran and Hussong, 2009). Numerous methods associated with IDA exist, including moderated nonlinear factor analysis (MNLFA; Bauer and Hussong, 2009), individual participant data meta-analysis (Cooper and Patall, 2009, Huh et al., 2015), and item response theory (McArdle et al., 2009). The primary requirements for IDA are to have multiple studies with common measures that reflect similar constructs (Bauer and Hussong, 2009).

In the current study we utilized MNLFA (Bauer and Hussong, 2009, Curran et al., 2014), which allowed for the empirical testing of whether items are measuring the same construct across subpopulations, including individuals from different studies and with different demographic characteristics. Tests of differential item functioning (DIF; e.g., items not measuring the same construct the same way across groups) can yield insights about the clinical value of certain items and item responses that may differentially reflect the construct in different groups.

The goal of this study was to examine whether the construct of alcohol dependence, as measured by different measures of alcohol dependence severity, was reproducible across studies. The pooling of data across studies provided greater statistical power and greater study heterogeneity to test for DIF. We had no a priori hypotheses regarding DIF given lack of prior research in this area among patients with AUD.

Section snippets

Source of data

The data for this study were drawn from four adult alcohol treatment studies (n = 4414). Exclusion criteria and participant demographics are provided in Table 1.

Item analysis

Graphical and descriptive analyses of the 23 AUI/ADS items, the DSM item, and the 10 LDQ items indicated there were similar frequencies of responses across studies. Evaluation of the item content suggested that thirteen of the AUI/ADS items and one of the LDQ items appeared to be measuring consequences of drinking (e.g., “passed out”) and level of alcohol consumption (e.g. “how much did you drink”). Given our ultimate goal of examining associations between alcohol dependence severity,

Discussion

This study used IDA to examine the alcohol dependence syndrome construct as assessed via the ADS, AUI, LDQ, and DSM in four different alcohol treatment studies. Consistent with prior work, we found support for the dimensionality and concurrent validity of the alcohol dependence construct (Edwards, 1986, Stockwell, 2015).

The current study revealed several differences between subgroups in the functioning of items measuring dependence severity. Seven of the 20 item intercepts and three of the 20

Role of funding source

The NIH/NIAAA had no role in the design and conduct of the study; management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Contributors

All authors had full access to the data and take responsibility for the integrity of the data. Dr. Witkiewitz conducted all analyses and takes responsibility for the accuracy of the data analysis. Dr. Hallgren, Mr. O'Sickey and Mr. Roos assisted with creating the combined datasets. Drs. Witkiewitz and Maisto were responsible for the study concept and design. Dr. Witkiewitz drafted the manuscript and all authors provided critical revisions of the manuscript.

Conflict of interest statement

Dr. Witkiewitz has served as a consultant for Alkermes and has received financial support from the American Society of Clinical Psychopharmacology's Alcohol Clinical Trials Initiative (ACTIVE), which is supported by AbbVie, Ethypharm, Lilly, Lundbeck, and Pfizer.

Acknowledgements

This research was supported by a National Institute on Alcohol Abuse and Alcoholism grant 1R01 AA022328 (to Katie Witkiewitz), a Senior Career Scientist Research and Mentoring Award 2K05 AA16928 (to Stephen A. Maisto, principal investigator), and by a training fellowship to Kevin Hallgren (1T32 AA007455-32, Mary E. Larimer principal investigator). We thank Dr. Daniel J. Bauer for his invaluable ongoing consultation on integrative data analysis and for editorial comments on this manuscript.

References (38)

  • C.W. Kahler et al.

    Item functioning of the alcohol dependence scale in a high-risk sample

    Drug Alcohol Depend.

    (2003)
  • B.T. Williams et al.

    The alcohol problems questionnaire: reliability and validity

    Drug Alcohol Depend.

    (1994)
  • American Psychiatric Association

    Diagnostic and Statistical Manual of Mental Disorders (4th Ed.; DSM-IV)

    (1994)
  • D.J. Bauer et al.

    Psychometric approaches for developing commensurate measures across independent studies: traditional and new models

    Psychol. Methods

    (2009)
  • P.M. Bentler

    Comparative fit indexes in structural models

    Psychol. Bull.

    (1990)
  • M.W. Browne et al.

    Alternative ways of assessing model fit

  • COMBINE Study Group

    Testing combined pharmacotherapies and behavioral interventions in alcohol dependence: rationale and methods

    Alcohol. Clin. Exp. Res.

    (2003)
  • H. Cooper et al.

    The relative benefits of meta-analysis conducted with individual participant data versus aggregated data

    Psychol. Methods

    (2009)
  • P.J. Curran et al.

    Integrative data analysis: the simultaneous analysis of multiple data sets

    Psychol. Methods

    (2009)
  • P.J. Curran et al.

    Pooling data from multiple longitudinal studies: the role of item response theory in integrative data analysis

    Dev. Psychol.

    (2008)
  • P.J. Curran et al.

    A moderated nonlinear factor model for the development of commensurate measures in integrative data analysis

    Multivariate Behav. Res.

    (2014)
  • S.R. Doyle et al.

    A validation study of the alcohol dependence scale

    J. Stud. Alcohol Drugs

    (2009)
  • G. Edwards

    The alcohol dependence syndrome: a concept as stimulus to enquiry

    Addiction

    (1986)
  • G. Edwards et al.

    Alcohol dependence: provisional description of a clinical syndrome

    Br. Med. J.

    (1976)
  • M.B. First et al.

    Structured clinical interview for DSM-IV-TR axis I disorders, research version, patient edition with psychotic screen

    Biometrics Research

    (2002)
  • J.L. Horn et al.

    Diagnosis of alcoholism. Factors of drinking, background and current conditions in alcoholics

    Q. J. Stud. Alcohol

    (1974)
  • J.L. Horn et al.

    Guide to the Alcohol Use Inventory (AUI)

    (1990)
  • D. Huh et al.

    Brief motivational interventions for college student drinking may not be as powerful as we think: an individual participant-level data meta-analysis

    Alcohol. Clin. Exp. Res.

    (2015)
  • J. Jager et al.

    Historical variation in drug use trajectories across the transition to adulthood: the trend toward lower intercepts and steeper, ascending slopes

    Dev. Psychopathol.

    (2013)
  • Cited by (21)

    • An application of moderated nonlinear factor analysis to develop a commensurate measure of alcohol problems across four alcohol treatment studies

      2021, Drug and Alcohol Dependence
      Citation Excerpt :

      Responses to the Form-90 were used to estimate percent heavy drinking days (PHDD), drinks per drinking day (DDD), and percent drinking days (PDD) at baseline and at 12-month follow-up, which occurred 12-months after baseline. Alcohol dependence severity was assessed at baseline using a commensurate measure developed by Witkiewitz et al. (2016) in a prior application of MNLFA with integrated data from these studies. Following recommendations of Curran et al. (2014), we used data from a calibration sample where observations for each participant are randomly selected from one of the time points to meet the assumption of independence of MNLFA that is violated with repeated measures data (Steps 1–3).

    • Evaluating treatments for posttraumatic stress disorder, alcohol and other drug use disorders using meta-analysis of individual patient data: Design and methodology of a virtual clinical trial

      2021, Contemporary Clinical Trials
      Citation Excerpt :

      As shown in the illustration in Table 1, the data structure for measures of PTSD will allow for multiple reports of PTSD symptoms per study, with most studies having symptom-level data from the CAPS in addition to one of the above-referenced self-report measures. Unlike a number of emerging IDA studies that use only one measure per study outcome, many of the datasets used in Project Harmony will have at least two measures of PTSD (e.g., both clinician-rated and self-reported), the items of which can be harmonized [23,51] to carefully consider estimation of scale scores in the presence of within-person symptom-level discordance between clinician and patient reporters [52] using data from both reporters in estimating latent underlying PTSD. AOD measures will be harmonized at the item-level, with emphasis on specific substances as noted in the measures section, with a similar harmonization structure to the PTSD measures to accommodate multiple AOD measures within study which >70% of the datasets have.

    • Gender-based differential item functioning in the Cannabis-Associated Problems Questionnaire: A replication and extension

      2021, Addictive Behaviors
      Citation Excerpt :

      Removal of biased items did not decrease internal consistency or convergent validity. Age- and gender-related DIF also appear in other measures of alcohol use and dependence, suggesting group differences on these scales require cautious interpretation (Witkiewitz, Hallgren, O'Sickey, Roos, & Maisto, 2016). Few studies address gender-biased DIF in measures of cannabis-related variables.

    • Simplifying the implementation of modern scale scoring methods with an automated R package: Automated moderated nonlinear factor analysis (aMNLFA)

      2019, Addictive Behaviors
      Citation Excerpt :

      By using our aMNLFA package, we have cut down on time and errors while retaining the benefits of using a high-quality scoring method (e.g., Gottfredson, Hussong, Ennett, & Rothenberg, 2017). With few exceptions (e.g., Smith, Rose, Mazure, Giovino, & McKee, 2014; Witkiewitz, Hallgren, O'Sickey, Roos, & Maisto, 2016), MNLFA has not been widely adopted. We hope that by formalizing aMNLFA into a readily-available and documented R package, and by illustrating its use here, other researchers will be able to experience these benefits.

    View all citing articles on Scopus
    View full text