Full length articleReproducibility and differential item functioning of the alcohol dependence syndrome construct across four alcohol treatment studies: An integrative data analysis
Introduction
The alcohol dependence construct is central to theories of problem alcohol use (Stockwell, 2015). However, the reproducibility of the alcohol dependence construct across studies and settings is not well known. There have been numerous psychometric studies of individual measures of alcohol dependence (Doyle and Donovan, 2009, Kahler et al., 2003a), but no studies have attempted to examine the construct of alcohol dependence across measures, studies, and patient subgroups. The goal of this study was to use integrative data analysis to test the construct of alcohol dependence severity among 4393 patients presenting for alcohol use disorder (AUD) treatment.
Edwards and Gross (1976) hypothesized that alcohol dependence syndrome was a collection of symptoms including narrowing of the drinking repertoire, salience of drink-seeking behavior, tolerance, drinking to relieve withdrawal symptoms, awareness of compulsion to drink, and return to drinking following abstinence. The dependence syndrome is primarily focused on physiological, psychological and behavioral symptoms that result in a core “drive to consume alcohol” (p. 79; Stockwell et al., 1979) and is therefore distinct from alcohol consumption, in and of itself, or negative consequences that result from alcohol use (Li et al., 2007). Almost 40 years of research on the dependence syndrome (Marshall, 2015, Stockwell, 2015) has supported the importance of the concept for the clinical formulation of AUD. Yet, the degree to which certain elements of the syndrome are central to the construct or subgroups (e.g., females) has received little attention (Kahler et al., 2003b), and no study has examined whether different measures of dependence are measuring the same construct across AUD samples.
IDA is a framework for merging, analyzing, and comparing information from multiple studies (Curran and Hussong, 2009). Numerous methods associated with IDA exist, including moderated nonlinear factor analysis (MNLFA; Bauer and Hussong, 2009), individual participant data meta-analysis (Cooper and Patall, 2009, Huh et al., 2015), and item response theory (McArdle et al., 2009). The primary requirements for IDA are to have multiple studies with common measures that reflect similar constructs (Bauer and Hussong, 2009).
In the current study we utilized MNLFA (Bauer and Hussong, 2009, Curran et al., 2014), which allowed for the empirical testing of whether items are measuring the same construct across subpopulations, including individuals from different studies and with different demographic characteristics. Tests of differential item functioning (DIF; e.g., items not measuring the same construct the same way across groups) can yield insights about the clinical value of certain items and item responses that may differentially reflect the construct in different groups.
The goal of this study was to examine whether the construct of alcohol dependence, as measured by different measures of alcohol dependence severity, was reproducible across studies. The pooling of data across studies provided greater statistical power and greater study heterogeneity to test for DIF. We had no a priori hypotheses regarding DIF given lack of prior research in this area among patients with AUD.
Section snippets
Source of data
The data for this study were drawn from four adult alcohol treatment studies (n = 4414). Exclusion criteria and participant demographics are provided in Table 1.
Item analysis
Graphical and descriptive analyses of the 23 AUI/ADS items, the DSM item, and the 10 LDQ items indicated there were similar frequencies of responses across studies. Evaluation of the item content suggested that thirteen of the AUI/ADS items and one of the LDQ items appeared to be measuring consequences of drinking (e.g., “passed out”) and level of alcohol consumption (e.g. “how much did you drink”). Given our ultimate goal of examining associations between alcohol dependence severity,
Discussion
This study used IDA to examine the alcohol dependence syndrome construct as assessed via the ADS, AUI, LDQ, and DSM in four different alcohol treatment studies. Consistent with prior work, we found support for the dimensionality and concurrent validity of the alcohol dependence construct (Edwards, 1986, Stockwell, 2015).
The current study revealed several differences between subgroups in the functioning of items measuring dependence severity. Seven of the 20 item intercepts and three of the 20
Role of funding source
The NIH/NIAAA had no role in the design and conduct of the study; management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Contributors
All authors had full access to the data and take responsibility for the integrity of the data. Dr. Witkiewitz conducted all analyses and takes responsibility for the accuracy of the data analysis. Dr. Hallgren, Mr. O'Sickey and Mr. Roos assisted with creating the combined datasets. Drs. Witkiewitz and Maisto were responsible for the study concept and design. Dr. Witkiewitz drafted the manuscript and all authors provided critical revisions of the manuscript.
Conflict of interest statement
Dr. Witkiewitz has served as a consultant for Alkermes and has received financial support from the American Society of Clinical Psychopharmacology's Alcohol Clinical Trials Initiative (ACTIVE), which is supported by AbbVie, Ethypharm, Lilly, Lundbeck, and Pfizer.
Acknowledgements
This research was supported by a National Institute on Alcohol Abuse and Alcoholism grant 1R01 AA022328 (to Katie Witkiewitz), a Senior Career Scientist Research and Mentoring Award 2K05 AA16928 (to Stephen A. Maisto, principal investigator), and by a training fellowship to Kevin Hallgren (1T32 AA007455-32, Mary E. Larimer principal investigator). We thank Dr. Daniel J. Bauer for his invaluable ongoing consultation on integrative data analysis and for editorial comments on this manuscript.
References (38)
- et al.
Item functioning of the alcohol dependence scale in a high-risk sample
Drug Alcohol Depend.
(2003) - et al.
The alcohol problems questionnaire: reliability and validity
Drug Alcohol Depend.
(1994) Diagnostic and Statistical Manual of Mental Disorders (4th Ed.; DSM-IV)
(1994)- et al.
Psychometric approaches for developing commensurate measures across independent studies: traditional and new models
Psychol. Methods
(2009) Comparative fit indexes in structural models
Psychol. Bull.
(1990)- et al.
Alternative ways of assessing model fit
Testing combined pharmacotherapies and behavioral interventions in alcohol dependence: rationale and methods
Alcohol. Clin. Exp. Res.
(2003)- et al.
The relative benefits of meta-analysis conducted with individual participant data versus aggregated data
Psychol. Methods
(2009) - et al.
Integrative data analysis: the simultaneous analysis of multiple data sets
Psychol. Methods
(2009) - et al.
Pooling data from multiple longitudinal studies: the role of item response theory in integrative data analysis
Dev. Psychol.
(2008)
A moderated nonlinear factor model for the development of commensurate measures in integrative data analysis
Multivariate Behav. Res.
A validation study of the alcohol dependence scale
J. Stud. Alcohol Drugs
The alcohol dependence syndrome: a concept as stimulus to enquiry
Addiction
Alcohol dependence: provisional description of a clinical syndrome
Br. Med. J.
Structured clinical interview for DSM-IV-TR axis I disorders, research version, patient edition with psychotic screen
Biometrics Research
Diagnosis of alcoholism. Factors of drinking, background and current conditions in alcoholics
Q. J. Stud. Alcohol
Guide to the Alcohol Use Inventory (AUI)
Brief motivational interventions for college student drinking may not be as powerful as we think: an individual participant-level data meta-analysis
Alcohol. Clin. Exp. Res.
Historical variation in drug use trajectories across the transition to adulthood: the trend toward lower intercepts and steeper, ascending slopes
Dev. Psychopathol.
Cited by (21)
An application of moderated nonlinear factor analysis to develop a commensurate measure of alcohol problems across four alcohol treatment studies
2021, Drug and Alcohol DependenceCitation Excerpt :Responses to the Form-90 were used to estimate percent heavy drinking days (PHDD), drinks per drinking day (DDD), and percent drinking days (PDD) at baseline and at 12-month follow-up, which occurred 12-months after baseline. Alcohol dependence severity was assessed at baseline using a commensurate measure developed by Witkiewitz et al. (2016) in a prior application of MNLFA with integrated data from these studies. Following recommendations of Curran et al. (2014), we used data from a calibration sample where observations for each participant are randomly selected from one of the time points to meet the assumption of independence of MNLFA that is violated with repeated measures data (Steps 1–3).
Evaluating treatments for posttraumatic stress disorder, alcohol and other drug use disorders using meta-analysis of individual patient data: Design and methodology of a virtual clinical trial
2021, Contemporary Clinical TrialsCitation Excerpt :As shown in the illustration in Table 1, the data structure for measures of PTSD will allow for multiple reports of PTSD symptoms per study, with most studies having symptom-level data from the CAPS in addition to one of the above-referenced self-report measures. Unlike a number of emerging IDA studies that use only one measure per study outcome, many of the datasets used in Project Harmony will have at least two measures of PTSD (e.g., both clinician-rated and self-reported), the items of which can be harmonized [23,51] to carefully consider estimation of scale scores in the presence of within-person symptom-level discordance between clinician and patient reporters [52] using data from both reporters in estimating latent underlying PTSD. AOD measures will be harmonized at the item-level, with emphasis on specific substances as noted in the measures section, with a similar harmonization structure to the PTSD measures to accommodate multiple AOD measures within study which >70% of the datasets have.
Gender-based differential item functioning in the Cannabis-Associated Problems Questionnaire: A replication and extension
2021, Addictive BehaviorsCitation Excerpt :Removal of biased items did not decrease internal consistency or convergent validity. Age- and gender-related DIF also appear in other measures of alcohol use and dependence, suggesting group differences on these scales require cautious interpretation (Witkiewitz, Hallgren, O'Sickey, Roos, & Maisto, 2016). Few studies address gender-biased DIF in measures of cannabis-related variables.
Simplifying the implementation of modern scale scoring methods with an automated R package: Automated moderated nonlinear factor analysis (aMNLFA)
2019, Addictive BehaviorsCitation Excerpt :By using our aMNLFA package, we have cut down on time and errors while retaining the benefits of using a high-quality scoring method (e.g., Gottfredson, Hussong, Ennett, & Rothenberg, 2017). With few exceptions (e.g., Smith, Rose, Mazure, Giovino, & McKee, 2014; Witkiewitz, Hallgren, O'Sickey, Roos, & Maisto, 2016), MNLFA has not been widely adopted. We hope that by formalizing aMNLFA into a readily-available and documented R package, and by illustrating its use here, other researchers will be able to experience these benefits.
Meta-analysis of DSM alcohol use disorder criteria severities: Structural consistency is only 'skin deep'
2016, Psychological Medicine