Elsevier

Drug and Alcohol Dependence

Volume 153, 1 August 2015, Pages 159-166
Drug and Alcohol Dependence

Comparison of provider-documented and patient-reported brief intervention for unhealthy alcohol use in VA outpatients

https://doi.org/10.1016/j.drugalcdep.2015.05.027Get rights and content

Highlights

  • We examined the association between provider EMR-documented and patient-reported brief intervention (BI).

  • We evaluated whether the association varied across 21 US Veterans Affairs networks.

  • There was high agreement between documented BI and patient-report of BI.

  • However, there was no variation in this relationship across VA networks.

  • Provider EMR-documentation of BI may be a valid method for assessing BI quality.

Abstract

Background

Performance measures for brief alcohol interventions (BIs) are currently based on provider documentation of BI. However, provider documentation may not be a reliable measure of whether or not patients are offered clinically meaningful BIs. In particular, BI documented with clinical decision support in an electronic medical record (EMR) could appear identical irrespective of the quality of BI provided. We hypothesized that differences in how BI was implemented across health systems could lead to differences in the proportion of documented BI recalled and reported by patients across health systems.

Methods

Male outpatients with unhealthy alcohol use identified by confidential satisfaction surveys (2009–2012) were assessed for whether they reported receiving BI in the past year (patient-reported BI) and whether they had BI documented in the EMR during the same period (documented BI). We evaluated and compared the prevalence of documented BI to patient-reported BI across 21 VA networks to determine whether documented BI had a variable association with patient-reported BI across the networks.

Results

Of 9896 eligible male outpatients with unhealthy alcohol use, 59.0% (95% CI 57.4–60.5%) reported BI (50.4–64.9% across networks) and 37.4% (95% CI 36.0–38.9%) had BI documented in the EMR (28.0–44.2% across networks). Overall, 72.9% (95% CI 70.8–75.5%) of patients with documented BI also reported BI. The association between documented BI and patient-reported BI did not vary across VA networks in adjusted logistic regression models.

Conclusions

Performance measures of BI that rely on provider documentation in EMRs appear comparable to patient report for comparing care across VA networks.

Introduction

Recent U.S. health care reform established alcohol screening and brief alcohol intervention (BI) as standard benefits (Health Care gov, 2013a, Health Care gov, 2013b). Yet, despite strong evidence for the efficacy of BI (Fleming et al., 1997, Jonas et al., 2012, Kaner et al., 2007, Ockene et al., 1999, Whitlock et al., 2004), efforts to integrate BI into routine medical settings have often failed (Nilsen, 2010, Nilsen, 2006, Solberg et al., 2008, Williams et al., 2011). Performance measures linked to incentives will likely play a central role in implementation of BI in health care systems in the US and elsewhere (Bradley et al., 2011a, Pincus et al., 2011).

To date, BI performance measures have been based on provider documentation of BI in the medical record, which can rely on electronic decision support or billing codes for BI (Centers for Medicare and Medicaid Services, 2011, Lapham et al., 2012, Physician Consortium for Performance Improvement, 2008, The Joint Commission, 2014). These measures are used to monitor and compare health care systems on the proportion of appropriate patients offered BI (Bradley et al., 2013). However, as for performance measurement of smoking cessation counseling and discharge planning (Chassin et al., 2010, Jha et al., 2009), in an era of electronic medical records (EMR) and performance monitoring, provider documentation may not accurately represent whether patients receive meaningful BIs. Performance incentives and feedback could lead providers to document minimal or perfunctory alcohol-related discussions as BIs, since EMR tools that facilitate documentation can make documentation of BI as easy as clicking a box. Such tools can make documentation identical irrespective of the quality of BI provided, such that a few seconds of alcohol-related discussion (McCormick et al., 2006) and 20 min of motivational interviewing could appear identical in the EMR. Because of the limitations of BI measures based on provider documentation, patient-report of BI on surveys has been used to evaluate receipt of BI following implementation efforts (Aalto et al., 2003, Babor et al., 2005, Chossis et al., 2007, Nilsen et al., 2011) and has been suggested as a preferable approach to measuring BIs (Bradley et al., 2013). However, provider documentation of BI has not been compared to an independent measure of BI, such as patient report, within the same sample of patients.

In 2008, the nationwide Veterans Affairs (VA) health care system implemented performance incentives for BI and nationally disseminated an electronic reminder in the EMR designed to prompt providers to offer BI and to facilitate documentation (Lapham et al., 2012, Williams et al., 2014). However, VA networks, the organizational level at which performance measurement was incentivized and benchmarked, were free to choose whether to use or modify the national BI reminder. Further, a previous study highlighted that alcohol screening in VA outpatient clinics – implemented like BI with performance measurement and incentives for network directors – identified fewer patients with unhealthy alcohol use than screening on confidential patient surveys (Bradley et al., 2011b) and that this relationship varied across networks. These results highlight the importance of accounting for network variation in the implementation of alcohol-related care practices in performance assessment.

The present study evaluated and compared documented BI to patient-reported BI in a single sample of patients across 21 VA networks during fiscal years (FY) 2009–2012 to determine if documented BI had a variable association with patient-reported BI across the networks. We hypothesized that the association between documented and patient-reported BI would differ across networks due to differing strategies of BI implementation. For example, networks that emphasized staff training and documentation of BI could have a different agreement between documented and patient-reported BI than networks that did not (Williams et al., 2013). If the association varied across networks, it would suggest that documented BI in the EMR might not be an acceptable performance measure for comparing implementation of BI across health care systems.

Section snippets

Study data

Secondary patient-level data from two national VA data sources were used for this study. Confidential data on patient-reported BI were obtained from the VA's Survey of Healthcare Experiences of Patients (SHEP), available through the VA Office of Performance Measurement within the Office of Analytics and Business Intelligence. The SHEP survey, mailed monthly to a national random sample of established patients with a recent outpatient visit (∼4.5% of outpatients per facility per month), provides

Study sample and patient characteristics

A total of 9896 male Veteran outpatients screened positive for unhealthy alcohol use on the survey AUDIT-C screen, completed the survey question about BI and had a clinical AUDIT-C screen documented in the EMR in the prior two years (Fig. 1). Survey weights were truncated at 500 for 101 patients; the median weight was 41.8 (interquartile range 25.9–81.7). This sample of patients who screened positive for unhealthy alcohol use on the SHEP survey was mostly white and older, with 39.4% having also

Discussion

In this sample of male VA outpatients who screened positive for unhealthy alcohol use on a confidential survey, nearly three-fourths (73%) of patients with documented BI recalled receiving BI. Further, our hypothesis that the association between provider documentation of BI and patient report of BI would vary across VA networks, due to variation in BI implementation, was not supported, suggesting that BI performance measures based on provider documented BI in the EMR were not influenced by

Role of funding source

The research reported here was funded by the National Institute on Alcohol Abuse and Alcoholism (1R21AA020894-01A1). Dr. Williams is supported by a Career Development Award from VA Health Services Research & Development (CDA 12-276) and is an investigator with the Implementation Research Institute (IRI) at the George Warren Brown School of Social Work at Washington University in St. Louis. IRI is supported through an award from the National Institute of Mental Health (R25 MH080916-01A2) and the

Contributors

Dr. Bradley designed the study and wrote the protocol. Dr. Lapham undertook the statistical analysis under the direction of Drs. Shortreed and Bradley, with input from all other coauthors. Dr. Lapham wrote the first draft of the manuscript, and all authors contributed to and have approved the final manuscript.

Conflict of interest statement

No conflicts of interests.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs, the United States Government, or any of the authors’ institutions.

Acknowledgments

We are grateful to the VA's Corporate Data Warehouse and the Office of Performance Measurement within the Office of Analytics and Business Intelligence for making their data available to this study.

References (77)

  • K.A. Bradley et al.

    When quality indicators undermine quality: bias in a quality indicator of follow-up for alcohol misuse

    Psychiatr. Serv.

    (2013)
  • K.A. Bradley et al.

    AUDIT-C as a brief screen for alcohol misuse in primary care

    Alcohol. Clin. Exp. Res.

    (2007)
  • K.A. Bradley et al.

    Commentary on Nilsen et al.: the importance of asking patients—the potential value of patient report of brief interventions

    Addiction

    (2011)
  • K.A. Bradley et al.

    Quality concerns with routine alcohol screening in VA clinical settings

    J. Gen. Intern. Med.

    (2011)
  • K.A. Bradley et al.

    Measuring performance of brief alcohol counseling in medical settings:a review of the options and lessons from the Veterans Affairs (VA) health care system

    Subst. Abuse

    (2007)
  • K.A. Bradley et al.

    Implementation of evidence-based alcohol screening in the Veterans Health Administration

    Am. J. Manag. Care

    (2006)
  • M.L. Burman et al.

    Alcohol-related advice for VA primary care patients: who gets it, who gives it?

    J. Stud. Alcohol

    (2004)
  • K. Bush et al.

    The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test

    Arch. Intern. Med.

    (1998)
  • Centers for Medicare and Medicaid Services

    Decision Memo for Screening and Behavioral Counseling Interventions in Primary Care to Reduce Alcohol Misuse (CAG-00427N)

    (2011)
  • M.R. Chassin et al.

    Accountability measures—using measurement to promote quality improvement

    N. Engl. J. Med.

    (2010)
  • L.J. Chavez et al.

    Increases in prevalence of patient-reported alcohol-related advice following performance measure implementation

  • I. Chossis et al.

    Effect of training on primary care residents’ performance in brief alcohol intervention: a randomized controlled trial

    J. Gen. Intern. Med.

    (2007)
  • S.R. Cole et al.

    Constructing inverse probability weights for marginal structural models

    Am. J. Epidemiol.

    (2008)
  • D.A. Dawson et al.

    Effectiveness of the derived Alcohol Use Disorders Identification Test (AUDIT-C) in screening for alcohol use disorders and risk drinking in the US general population

    Alcohol. Clin. Exp. Res.

    (2005)
  • J.G. Demakis et al.

    Improving residents’ compliance with standards of ambulatory care: results from the VA cooperative study on computerized reminders

    JAMA

    (2000)
  • M.F. Fleming et al.

    Brief physician advice for problem alcohol drinkers: a randomized controlled trial in community-based primary care practices

    JAMA

    (1997)
  • D. Frank et al.

    Effectiveness of the AUDIT-C as a screening test for alcohol misuse in three race/ethnic groups

    J. Gen. Intern. Med.

    (2008)
  • C.H. Fung et al.

    An evaluation of the Veterans Health Administration's clinical reminders system: a national survey of generalists

    J. Gen. Intern. Med.

    (2008)
  • C.H. Fung et al.

    Variation in implementation and use of computerized clinical reminders in an integrated healthcare system

    Am. J. Manag. Care

    (2004)
  • G. Gmel

    The effect of mode of data collection and of non-response on reported alcohol consumption: a split-sample study in Switzerland

    Addiction

    (2000)
  • T. Hastie et al.

    The Elements of Statistical Learning

    (2009)
  • E.J. Hawkins et al.

    Examining quality issues in alcohol misuse screening

    Subst. Abuse

    (2007)
  • HealthCare.gov

    What Are My Preventive Care Benefits?

    (2013)
  • HealthCare.gov

    What Does Marketplace Health Insurance Cover?

    (2013)
  • A.K. Jha et al.

    Public reporting of discharge planning and rates of readmissions

    N. Engl. J. Med.

    (2009)
  • D.E. Jonas et al.

    Behavioral counseling after screening for alcohol misuse in primary care: a systematic review and meta-analysis for the U.S. Preventive Services Task Force

    Ann. Intern. Med.

    (2012)
  • E.F. Kaner et al.

    Effectiveness of brief alcohol interventions in primary care populations

    Cochrane Database Syst. Rev.

    (2007)
  • E.F. Kaner et al.

    Patient and practitioner characteristics predict brief alcohol intervention in primary care

    Br. J. Gen. Pract.

    (2001)
  • Cited by (16)

    • Racial/ethnic and gender differences in receipt of brief intervention among patients with unhealthy alcohol use in the U.S. Veterans Health Administration

      2020, Journal of Substance Abuse Treatment
      Citation Excerpt :

      Consistent with prior studies (Bensley et al., 2019; Chen et al., 2018; Owens et al., 2018; Williams, Lapham, Andersen, et al., 2017; Williams, Lapham, Bobb, et al., 2017), we extracted our measure of BI from text data called “health factors” that are generated from the VA's clinical decision support tool (“clinical reminder”), a templated form used to prompt and document BI (Williams et al., 2014). This provider-documented, EHR text-based measure of BI has been found to have good concordance (73%) with patient-reported BI (Lapham et al., 2015) and is similar to methods used to evaluate implementation of BI in non-VA healthcare systems (Chi et al., 2017; Slain et al., 2014). We extracted covariates from the EHR and selected them a priori based on the expectation that they would influence the outcome but not be causally related to race/ethnicity or gender (Alvanzo et al., 2014; Dobscha et al., 2009; Glass et al., 2016; Hebert et al., 2008; Williams et al., 2012).

    • Receipt of alcohol-related care among patients with HCV and unhealthy alcohol use

      2018, Drug and Alcohol Dependence
      Citation Excerpt :

      This measure is consistent with components included in efficacious brief interventions (Whitlock et al., 2004) and with VA’s national performance measure, which requires documentation of advice to reduce or abstain from drinking and feedback linking drinking to health for all patients who screen positive on the AUDIT-C with scores ≥5 within 14 days of screening positive (Lapham et al., 2012; Williams et al., 2017b; Williams et al., 2014). Similar to previous studies (Bradley et al., 2013; Lapham et al., 2015; Williams et al., 2014), the measure of brief intervention was derived from data that result from documentation of advice to reduce or abstain from drinking in an electronic clinical reminder. This becomes “due” for providers once a patient screens positive on the AUDIT-C and is routinely used across VA sites to facilitate meeting VA’s performance measure (Lapham et al., 2012; Williams et al., 2014).

    • Alcohol-related and mental health care for patients with unhealthy alcohol use and posttraumatic stress disorder in a National Veterans Affairs cohort

      2018, Journal of Substance Abuse Treatment
      Citation Excerpt :

      Brief interventions were defined as documented advice to reduce or abstain from drinking within 14 days following the index positive alcohol screen, consistent with VA's performance measure for evidence-based brief interventions (Lapham et al., 2015; Williams et al., 2014). As has been done previously, this measure was derived using EHR text data generated by documentation in VA's clinical decision support tool (“clinical reminder”) for brief interventions (Lapham et al., 2015; Williams et al., 2014). Specialty addictions treatment for AUD was defined as any encounter associated with an AUD diagnosis in the specialty addictions inpatient or outpatient setting 0–365 days following the index positive screen (Appendix A).

    • Documented brief intervention not associated with resolution of unhealthy alcohol use one year later among VA patients living with HIV

      2017, Journal of Substance Abuse Treatment
      Citation Excerpt :

      Any documented brief intervention—the primary predictor variable—was defined as documentation of advice to reduce and/or abstain from drinking in the 0–14 days following a positive alcohol screen (Lapham et al., 2015). This component of VA's performance measure for brief intervention is routinely documented using electronic clinical reminders, resulting in text data that can be extracted from the CDW to measure its receipt (Lapham et al., 2015; Williams, Achtmeyer et al., 2016; Williams et al., 2014). Because repeated brief interventions may be more effective than a single brief intervention (Jonas et al., 2012) we used a categorical secondary predictor variable based on the number of brief interventions documented in the 0–365 days after a positive screen.

    View all citing articles on Scopus

    Supplementary material can be found by accessing the online version of this paper at http://dx.doi.org and by entering doi: 10.1016/j.drugalcdep.2015.05.027.

    View full text