Comparison of provider-documented and patient-reported brief intervention for unhealthy alcohol use in VA outpatients☆
Introduction
Recent U.S. health care reform established alcohol screening and brief alcohol intervention (BI) as standard benefits (Health Care gov, 2013a, Health Care gov, 2013b). Yet, despite strong evidence for the efficacy of BI (Fleming et al., 1997, Jonas et al., 2012, Kaner et al., 2007, Ockene et al., 1999, Whitlock et al., 2004), efforts to integrate BI into routine medical settings have often failed (Nilsen, 2010, Nilsen, 2006, Solberg et al., 2008, Williams et al., 2011). Performance measures linked to incentives will likely play a central role in implementation of BI in health care systems in the US and elsewhere (Bradley et al., 2011a, Pincus et al., 2011).
To date, BI performance measures have been based on provider documentation of BI in the medical record, which can rely on electronic decision support or billing codes for BI (Centers for Medicare and Medicaid Services, 2011, Lapham et al., 2012, Physician Consortium for Performance Improvement, 2008, The Joint Commission, 2014). These measures are used to monitor and compare health care systems on the proportion of appropriate patients offered BI (Bradley et al., 2013). However, as for performance measurement of smoking cessation counseling and discharge planning (Chassin et al., 2010, Jha et al., 2009), in an era of electronic medical records (EMR) and performance monitoring, provider documentation may not accurately represent whether patients receive meaningful BIs. Performance incentives and feedback could lead providers to document minimal or perfunctory alcohol-related discussions as BIs, since EMR tools that facilitate documentation can make documentation of BI as easy as clicking a box. Such tools can make documentation identical irrespective of the quality of BI provided, such that a few seconds of alcohol-related discussion (McCormick et al., 2006) and 20 min of motivational interviewing could appear identical in the EMR. Because of the limitations of BI measures based on provider documentation, patient-report of BI on surveys has been used to evaluate receipt of BI following implementation efforts (Aalto et al., 2003, Babor et al., 2005, Chossis et al., 2007, Nilsen et al., 2011) and has been suggested as a preferable approach to measuring BIs (Bradley et al., 2013). However, provider documentation of BI has not been compared to an independent measure of BI, such as patient report, within the same sample of patients.
In 2008, the nationwide Veterans Affairs (VA) health care system implemented performance incentives for BI and nationally disseminated an electronic reminder in the EMR designed to prompt providers to offer BI and to facilitate documentation (Lapham et al., 2012, Williams et al., 2014). However, VA networks, the organizational level at which performance measurement was incentivized and benchmarked, were free to choose whether to use or modify the national BI reminder. Further, a previous study highlighted that alcohol screening in VA outpatient clinics – implemented like BI with performance measurement and incentives for network directors – identified fewer patients with unhealthy alcohol use than screening on confidential patient surveys (Bradley et al., 2011b) and that this relationship varied across networks. These results highlight the importance of accounting for network variation in the implementation of alcohol-related care practices in performance assessment.
The present study evaluated and compared documented BI to patient-reported BI in a single sample of patients across 21 VA networks during fiscal years (FY) 2009–2012 to determine if documented BI had a variable association with patient-reported BI across the networks. We hypothesized that the association between documented and patient-reported BI would differ across networks due to differing strategies of BI implementation. For example, networks that emphasized staff training and documentation of BI could have a different agreement between documented and patient-reported BI than networks that did not (Williams et al., 2013). If the association varied across networks, it would suggest that documented BI in the EMR might not be an acceptable performance measure for comparing implementation of BI across health care systems.
Section snippets
Study data
Secondary patient-level data from two national VA data sources were used for this study. Confidential data on patient-reported BI were obtained from the VA's Survey of Healthcare Experiences of Patients (SHEP), available through the VA Office of Performance Measurement within the Office of Analytics and Business Intelligence. The SHEP survey, mailed monthly to a national random sample of established patients with a recent outpatient visit (∼4.5% of outpatients per facility per month), provides
Study sample and patient characteristics
A total of 9896 male Veteran outpatients screened positive for unhealthy alcohol use on the survey AUDIT-C screen, completed the survey question about BI and had a clinical AUDIT-C screen documented in the EMR in the prior two years (Fig. 1). Survey weights were truncated at 500 for 101 patients; the median weight was 41.8 (interquartile range 25.9–81.7). This sample of patients who screened positive for unhealthy alcohol use on the SHEP survey was mostly white and older, with 39.4% having also
Discussion
In this sample of male VA outpatients who screened positive for unhealthy alcohol use on a confidential survey, nearly three-fourths (73%) of patients with documented BI recalled receiving BI. Further, our hypothesis that the association between provider documentation of BI and patient report of BI would vary across VA networks, due to variation in BI implementation, was not supported, suggesting that BI performance measures based on provider documented BI in the EMR were not influenced by
Role of funding source
The research reported here was funded by the National Institute on Alcohol Abuse and Alcoholism (1R21AA020894-01A1). Dr. Williams is supported by a Career Development Award from VA Health Services Research & Development (CDA 12-276) and is an investigator with the Implementation Research Institute (IRI) at the George Warren Brown School of Social Work at Washington University in St. Louis. IRI is supported through an award from the National Institute of Mental Health (R25 MH080916-01A2) and the
Contributors
Dr. Bradley designed the study and wrote the protocol. Dr. Lapham undertook the statistical analysis under the direction of Drs. Shortreed and Bradley, with input from all other coauthors. Dr. Lapham wrote the first draft of the manuscript, and all authors contributed to and have approved the final manuscript.
Conflict of interest statement
No conflicts of interests.
Disclaimer
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, the United States Government, or any of the authors’ institutions.
Acknowledgments
We are grateful to the VA's Corporate Data Warehouse and the Office of Performance Measurement within the Office of Analytics and Business Intelligence for making their data available to this study.
References (77)
- et al.
Primary health care professionals’ activity in intervening in patients’ alcohol drinking during a 3-year brief intervention implementation project
Drug Alcohol Depend.
(2003) - et al.
Ten commandments for effective clinical decision support: making the practice of evidence-based medicine a reality
JAMA
(2003) - et al.
Physician advice about smoking and drinking: are U.S. adults being informed?
Am. J. Prev. Med.
(2003) - et al.
Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases
J. Clin. Epidemiol.
(1992) - et al.
Decreasing sensitivity of clinical alcohol screening with the AUDIT-C after repeated negative screens in VA clinics
Drug Alcohol Depend.
(2014) - et al.
Identifying barriers to the effective use of clinical reminders: bootstrapping multiple methods
J. Biomed. Inf.
(2005) - et al.
Exploring barriers and facilitators to the use of computerized clinical reminders
J. Am. Med. Inf. Assoc.
(2005) - et al.
Alcohol screening and brief intervention in primary care settings: implementation models and predictors
J. Stud. Alcohol
(2005) Mode of questionnaire administration can have serious effects on data quality
J. Public Health (Oxford)
(2005)- et al.
Screening for problem drinking: comparison of CAGE and AUDIT
J. Gen. Intern. Med.
(1998)
When quality indicators undermine quality: bias in a quality indicator of follow-up for alcohol misuse
Psychiatr. Serv.
AUDIT-C as a brief screen for alcohol misuse in primary care
Alcohol. Clin. Exp. Res.
Commentary on Nilsen et al.: the importance of asking patients—the potential value of patient report of brief interventions
Addiction
Quality concerns with routine alcohol screening in VA clinical settings
J. Gen. Intern. Med.
Measuring performance of brief alcohol counseling in medical settings:a review of the options and lessons from the Veterans Affairs (VA) health care system
Subst. Abuse
Implementation of evidence-based alcohol screening in the Veterans Health Administration
Am. J. Manag. Care
Alcohol-related advice for VA primary care patients: who gets it, who gives it?
J. Stud. Alcohol
The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test
Arch. Intern. Med.
Decision Memo for Screening and Behavioral Counseling Interventions in Primary Care to Reduce Alcohol Misuse (CAG-00427N)
Accountability measures—using measurement to promote quality improvement
N. Engl. J. Med.
Increases in prevalence of patient-reported alcohol-related advice following performance measure implementation
Effect of training on primary care residents’ performance in brief alcohol intervention: a randomized controlled trial
J. Gen. Intern. Med.
Constructing inverse probability weights for marginal structural models
Am. J. Epidemiol.
Effectiveness of the derived Alcohol Use Disorders Identification Test (AUDIT-C) in screening for alcohol use disorders and risk drinking in the US general population
Alcohol. Clin. Exp. Res.
Improving residents’ compliance with standards of ambulatory care: results from the VA cooperative study on computerized reminders
JAMA
Brief physician advice for problem alcohol drinkers: a randomized controlled trial in community-based primary care practices
JAMA
Effectiveness of the AUDIT-C as a screening test for alcohol misuse in three race/ethnic groups
J. Gen. Intern. Med.
An evaluation of the Veterans Health Administration's clinical reminders system: a national survey of generalists
J. Gen. Intern. Med.
Variation in implementation and use of computerized clinical reminders in an integrated healthcare system
Am. J. Manag. Care
The effect of mode of data collection and of non-response on reported alcohol consumption: a split-sample study in Switzerland
Addiction
The Elements of Statistical Learning
Examining quality issues in alcohol misuse screening
Subst. Abuse
What Are My Preventive Care Benefits?
What Does Marketplace Health Insurance Cover?
Public reporting of discharge planning and rates of readmissions
N. Engl. J. Med.
Behavioral counseling after screening for alcohol misuse in primary care: a systematic review and meta-analysis for the U.S. Preventive Services Task Force
Ann. Intern. Med.
Effectiveness of brief alcohol interventions in primary care populations
Cochrane Database Syst. Rev.
Patient and practitioner characteristics predict brief alcohol intervention in primary care
Br. J. Gen. Pract.
Cited by (16)
Effect of a family planning program on documented emotional support and reproductive goals counseling after previable pregnancy loss
2022, Patient Education and CounselingRacial/ethnic and gender differences in receipt of brief intervention among patients with unhealthy alcohol use in the U.S. Veterans Health Administration
2020, Journal of Substance Abuse TreatmentCitation Excerpt :Consistent with prior studies (Bensley et al., 2019; Chen et al., 2018; Owens et al., 2018; Williams, Lapham, Andersen, et al., 2017; Williams, Lapham, Bobb, et al., 2017), we extracted our measure of BI from text data called “health factors” that are generated from the VA's clinical decision support tool (“clinical reminder”), a templated form used to prompt and document BI (Williams et al., 2014). This provider-documented, EHR text-based measure of BI has been found to have good concordance (73%) with patient-reported BI (Lapham et al., 2015) and is similar to methods used to evaluate implementation of BI in non-VA healthcare systems (Chi et al., 2017; Slain et al., 2014). We extracted covariates from the EHR and selected them a priori based on the expectation that they would influence the outcome but not be causally related to race/ethnicity or gender (Alvanzo et al., 2014; Dobscha et al., 2009; Glass et al., 2016; Hebert et al., 2008; Williams et al., 2012).
Receipt of alcohol-related care among patients with HCV and unhealthy alcohol use
2018, Drug and Alcohol DependenceCitation Excerpt :This measure is consistent with components included in efficacious brief interventions (Whitlock et al., 2004) and with VA’s national performance measure, which requires documentation of advice to reduce or abstain from drinking and feedback linking drinking to health for all patients who screen positive on the AUDIT-C with scores ≥5 within 14 days of screening positive (Lapham et al., 2012; Williams et al., 2017b; Williams et al., 2014). Similar to previous studies (Bradley et al., 2013; Lapham et al., 2015; Williams et al., 2014), the measure of brief intervention was derived from data that result from documentation of advice to reduce or abstain from drinking in an electronic clinical reminder. This becomes “due” for providers once a patient screens positive on the AUDIT-C and is routinely used across VA sites to facilitate meeting VA’s performance measure (Lapham et al., 2012; Williams et al., 2014).
Alcohol-related and mental health care for patients with unhealthy alcohol use and posttraumatic stress disorder in a National Veterans Affairs cohort
2018, Journal of Substance Abuse TreatmentCitation Excerpt :Brief interventions were defined as documented advice to reduce or abstain from drinking within 14 days following the index positive alcohol screen, consistent with VA's performance measure for evidence-based brief interventions (Lapham et al., 2015; Williams et al., 2014). As has been done previously, this measure was derived using EHR text data generated by documentation in VA's clinical decision support tool (“clinical reminder”) for brief interventions (Lapham et al., 2015; Williams et al., 2014). Specialty addictions treatment for AUD was defined as any encounter associated with an AUD diagnosis in the specialty addictions inpatient or outpatient setting 0–365 days following the index positive screen (Appendix A).
Documented brief intervention not associated with resolution of unhealthy alcohol use one year later among VA patients living with HIV
2017, Journal of Substance Abuse TreatmentCitation Excerpt :Any documented brief intervention—the primary predictor variable—was defined as documentation of advice to reduce and/or abstain from drinking in the 0–14 days following a positive alcohol screen (Lapham et al., 2015). This component of VA's performance measure for brief intervention is routinely documented using electronic clinical reminders, resulting in text data that can be extracted from the CDW to measure its receipt (Lapham et al., 2015; Williams, Achtmeyer et al., 2016; Williams et al., 2014). Because repeated brief interventions may be more effective than a single brief intervention (Jonas et al., 2012) we used a categorical secondary predictor variable based on the number of brief interventions documented in the 0–365 days after a positive screen.
- ☆
Supplementary material can be found by accessing the online version of this paper at http://dx.doi.org and by entering doi: 10.1016/j.drugalcdep.2015.05.027.