Elsevier

Drug and Alcohol Dependence

Volume 154, 1 September 2015, Pages 300-303
Drug and Alcohol Dependence

Short communication
Examination of a recommended algorithm for eliminating nonsystematic delay discounting response sets

https://doi.org/10.1016/j.drugalcdep.2015.07.011Get rights and content

Highlights

  • Nonsystematic delay discounting response sets were identified using an algorithm.

  • The algorithm removes fewer cases than the conventional R2 statistic.

  • Removing nonsystematic response sets did not change the relationship between discounting and quitting smoking in this data set.

  • Younger age and lower educational attainment both predict provision of nonsystematic data sets.

Abstract

Purpose

To examine (1) whether use of a recommended algorithm (Johnson and Bickel, 2008) improves upon conventional statistical model fit (R2) for identifying nonsystematic response sets in delay discounting (DD) data, (2) whether removing such data meaningfully effects research outcomes, and (3) to identify participant characteristics associated with nonsystematic response sets.

Methods

Discounting of hypothetical monetary rewards was assessed among 349 pregnant women (231 smokers and 118 recent quitters) via a computerized task comparing $1000 at seven future time points with smaller values available immediately. Nonsystematic response sets were identified using the algorithm and conventional statistical model fit (R2). The association between DD and quitting was analyzed with and without nonsystematic response sets to examine whether the inclusion or exclusion impacts this relationship. Logistic regression was used to examine whether participant sociodemographics were associated with nonsystematic response sets.

Results

The algorithm excluded fewer cases than the R2 method (14% vs. 16%), and was not correlated with log k as is R2. The relationship between log k and the clinical outcome (spontaneous quitting) was unaffected by exclusion methods; however, other variables in the model were affected. Lower educational attainment and younger age were associated with nonsystematic response sets.

Conclusions

The algorithm eliminated data that were inconsistent with the nature of discounting and retained data that were orderly. Neither method impacted the smoking/DD relationship in this data set. Nonsystematic response sets are more likely among younger and less educated participants, who may need extra training or support in DD studies.

Introduction

Delay discounting (DD) is a behavioral-economic concept that measures reductions in the subjective value of consequences as a function of temporal delays to their delivery, with decrements in subjective value occurring relatively rapidly at shorter delays, and then less rapidly as delays become longer (Bickel and Marsch, 2001, Critchfield and Kollins, 2001). Some questions remain whether DD is a stable trait variable representing a type of impulsivity (Loewenstein et al., 2007) or personality characteristic (Odum, 2011) or whether the rate of discounting can be manipulated (Bickel et al., 2012). Because a number of interventions have been shown to alter discounting rate (Koffarnus et al., 2013), DD may prove to be an important target for treatment of the myriad of health-related risk behaviors and associated problems with which it is associated, including smoking and other substance abuse, HIV and other sexual risk behavior, pathological gambling, eating disorders, bipolar disorder, and adherence with disease prevention regimens (Bickel et al., 2012, Bickel and Marsch, 2001, Bradford, 2010, Davis et al., 2010, Herrmann et al., 2014, MacKillop et al., 2011, Rogers et al., 2010).

One practical challenge when attempting to predict real-world behavior using a self-report laboratory task is determining whether a subject's responses are conceptually interpretable (Critchfield and Kollins, 2001). DD research has incorporated the statistic R2 from linear regression to measure deviation from the hyperbolic curve; this statistic is typically described as measuring goodness-of-fit in delay discounting research (e.g., Green and Myerson, 1996, Myerson and Green, 1995, Ohmura et al., 2006, Rachlin et al., 1991). However, the use of R2 has significant problems. In general, the use of model fit as a way to assess data orderliness is itself limited because it imposes the presumption of a specific model onto data. Additionally and independently, R2 is systematically confounded because it uses a flat function as the comparator model, which means that R2 becomes systematically more stringent at lower discounting rates because the discounting function itself starts to resemble the comparator model. This means that R2 is biased toward labeling steeper (more impulsive) discounting curves as inherently better fitting than shallower discounting curves, which introduces a systematic bias against shallow discounters (Johnson and Bickel, 2008).

Johnson and Bickel (2008) strongly recommended the use of algorithms that test basic assumptions of discounting data (namely that the immediate value of rewards should diminish as the delay to receiving those rewards increases, and that this process is unidirectional, i.e. that the immediate value of delayed rewards only diminishes as delay increases). In this paper (Johnson and Bickel, 2008), the authors proposed and tested two criteria that were examples of the algorithmic approach. These example criteria (called Rule 1 and Rule 2 for the rest of this paper) were defined as follows: in Rule 1, cases in which any indifference point was greater than its predecessor by a magnitude exceeding 20% of the larger later reward (e.g., an increase of over $200 between two sequential time points on a task in which the large later reward is $1000); in Rule 2, if the indifference point at the final time point was not lower than the first time point by at least 10% of the large later reward (e.g., when the large later reward is $1000, the indifference point at the farthest time point is less than $100 smaller than the indifference point at the closest time point). The present study examines the relative merits of Johnson and Bickel's algorithm and conventional statistical model fit for characterizing DD data. Although data elimination was not the primary focus of the 2008 article, the current report examines how a DD data set is affected by eliminating data via the use of algorithms and statistical model fit, which were operationalized here as Rule 1 and Rule 2 vs. multiple cutoffs of R2 (0, 0.5 and 0.9). Like the algorithms, these cutoffs are not set in stone; various cutoffs can be found in the literature (e.g., 0.3, 0.4, etc.). In the current paper, three questions are asked: first, what percentage of a data set is eliminated when using these methods? Second, do these methods of eliminating data lead to meaningful outcome changes when DD is used as a predictor of smoking status? Third, do basic sociodemographic characteristics predict nonsystematic response sets on a computerized DD task?

Section snippets

Participants

Participants were 349 pregnant women who were regular smokers upon learning of their pregnancy: 118 of them quit smoking prior to initiating prenatal care, and 231 were still smoking at the start of prenatal care. Smoking status for all participants was biochemically verified using urine cotinine testing. Participants were recruited from obstetrical clinics located in the greater Burlington, Vermont area. The University of Vermont College of Medicine's Institutional Review Board approved these

Results

The use of the algorithm to exclude data sets resulted in 14% (49/349) of cases being excluded (Table 1): Rule 1 eliminated 25 cases, Rule 2 eliminated 32 cases, and the use of both criteria eliminated 49 cases (i.e., 8 participants met exclusion criteria under both rules). The use of R2 excluded 16% (55/349) of cases; 45 were excluded because R2 < 0, and 10 were removed because indifference points were identical across all seven time points (flat response set) and so a R2 statistic could not be

Discussion

This study aimed to systematically examine the differential impact of two methods for characterizing the orderliness of DD data: the algorithmic method and conventional statistical model fit. DD data were systematically eliminated from a large data set of data from pregnant smokers, using various combinations of the methods, to see the effects on the data set. We found that in our data set (349 pregnant women who were smoking when they learned of pregnancy), a large percentage of subjects (14%)

Role of funding source

Nothing declared.

Contributors

Dr. White and Dr. Redner devised the concept for the paper and proposed the analyses; Dr. White wrote the manuscript. Ms. Skelly conducted the data analyses and wrote the statistical methods section. She approved the statistical language in the results section and tables, and provided feedback on the manuscript. Dr. Higgins was the lab director and principal investigator for the pregnant smokers project from which this data set derived. He was also the postdoctoral mentor to Dr. White and Dr.

Conflicts of interest

None.

Acknowledgements

This research was supported by National Institutes of Health Center of Biomedical Research Excellence award P20GM103644 from the National Institute of General Medical Sciences, Tobacco Centers of Regulatory Science award P50DA036114 from the National Institute on Drug Abuse and U.S. Food and Drug Administration, research grants R01DA14028 and R01HD075669 from the National Institute on Drug Abuse and National Institute of Child Health and Human Development, respectively, and Institutional

References (19)

There are more references available in the full text version of this article.

Cited by (6)

  • A review of tobacco regulatory science research on vulnerable populations

    2019, Preventive Medicine
    Citation Excerpt :

    The fifth study provides novel cross-sectional data underscoring that use of high nicotine-yield cigarettes among women of reproductive age predicts tobacco dependence and smoking through pregnancy (Higgins et al., 2017d). Those five reports utilizing nationally-representative samples are complemented by studies with convenience samples (a) providing seminal data on the timing of smoking cessation during pregnancy (Heil et al., 2014), (b) using socioeconomic characteristics and innovative behavioral-economic tasks to predict smoking cessation and e-cigarette use (Chivers et al., 2016; White et al., 2014; White et al., 2015; Higgins et al., 2017e), and (c) providing new knowledge on the time-course of pregnancy-related changes in nicotine metabolism (Taghavi et al., 2018a; Taghavi et al., 2018b) and experimental evidence suggesting that those changes do not promote compensatory smoking (Bergeria et al., 2018). Another group of reports (Higgins et al., 2017b; Arger et al., 2017; Higgins et al., 2017c; Higgins et al., 2018) under the behavior, addiction, and impact analysis domains provided experimental evidence that VLNCs decrease the addiction potential of cigarette smoking among women of reproductive age, complementing results on VLNCs in other populations (Donny et al., 2015).

  • Effects of delay and probability combinations on discounting in humans

    2016, Behavioural Processes
    Citation Excerpt :

    If the criteria were met, all data sets from that participant were removed from the analysis. This algorithm for eliminating nonsystematic delay discounting responses was chosen based on recent findings the algorithm excludes fewer cases and is uncorrelated with log k values (White et al., 2015). The above data exclusion criteria reduced the total number of response sets from 212 to 172.

  • Interaction Design of Mobile-Based Product Display System

    2022, Mathematical Problems in Engineering
View full text