Short communicationExamination of a recommended algorithm for eliminating nonsystematic delay discounting response sets
Introduction
Delay discounting (DD) is a behavioral-economic concept that measures reductions in the subjective value of consequences as a function of temporal delays to their delivery, with decrements in subjective value occurring relatively rapidly at shorter delays, and then less rapidly as delays become longer (Bickel and Marsch, 2001, Critchfield and Kollins, 2001). Some questions remain whether DD is a stable trait variable representing a type of impulsivity (Loewenstein et al., 2007) or personality characteristic (Odum, 2011) or whether the rate of discounting can be manipulated (Bickel et al., 2012). Because a number of interventions have been shown to alter discounting rate (Koffarnus et al., 2013), DD may prove to be an important target for treatment of the myriad of health-related risk behaviors and associated problems with which it is associated, including smoking and other substance abuse, HIV and other sexual risk behavior, pathological gambling, eating disorders, bipolar disorder, and adherence with disease prevention regimens (Bickel et al., 2012, Bickel and Marsch, 2001, Bradford, 2010, Davis et al., 2010, Herrmann et al., 2014, MacKillop et al., 2011, Rogers et al., 2010).
One practical challenge when attempting to predict real-world behavior using a self-report laboratory task is determining whether a subject's responses are conceptually interpretable (Critchfield and Kollins, 2001). DD research has incorporated the statistic R2 from linear regression to measure deviation from the hyperbolic curve; this statistic is typically described as measuring goodness-of-fit in delay discounting research (e.g., Green and Myerson, 1996, Myerson and Green, 1995, Ohmura et al., 2006, Rachlin et al., 1991). However, the use of R2 has significant problems. In general, the use of model fit as a way to assess data orderliness is itself limited because it imposes the presumption of a specific model onto data. Additionally and independently, R2 is systematically confounded because it uses a flat function as the comparator model, which means that R2 becomes systematically more stringent at lower discounting rates because the discounting function itself starts to resemble the comparator model. This means that R2 is biased toward labeling steeper (more impulsive) discounting curves as inherently better fitting than shallower discounting curves, which introduces a systematic bias against shallow discounters (Johnson and Bickel, 2008).
Johnson and Bickel (2008) strongly recommended the use of algorithms that test basic assumptions of discounting data (namely that the immediate value of rewards should diminish as the delay to receiving those rewards increases, and that this process is unidirectional, i.e. that the immediate value of delayed rewards only diminishes as delay increases). In this paper (Johnson and Bickel, 2008), the authors proposed and tested two criteria that were examples of the algorithmic approach. These example criteria (called Rule 1 and Rule 2 for the rest of this paper) were defined as follows: in Rule 1, cases in which any indifference point was greater than its predecessor by a magnitude exceeding 20% of the larger later reward (e.g., an increase of over $200 between two sequential time points on a task in which the large later reward is $1000); in Rule 2, if the indifference point at the final time point was not lower than the first time point by at least 10% of the large later reward (e.g., when the large later reward is $1000, the indifference point at the farthest time point is less than $100 smaller than the indifference point at the closest time point). The present study examines the relative merits of Johnson and Bickel's algorithm and conventional statistical model fit for characterizing DD data. Although data elimination was not the primary focus of the 2008 article, the current report examines how a DD data set is affected by eliminating data via the use of algorithms and statistical model fit, which were operationalized here as Rule 1 and Rule 2 vs. multiple cutoffs of R2 (0, 0.5 and 0.9). Like the algorithms, these cutoffs are not set in stone; various cutoffs can be found in the literature (e.g., 0.3, 0.4, etc.). In the current paper, three questions are asked: first, what percentage of a data set is eliminated when using these methods? Second, do these methods of eliminating data lead to meaningful outcome changes when DD is used as a predictor of smoking status? Third, do basic sociodemographic characteristics predict nonsystematic response sets on a computerized DD task?
Section snippets
Participants
Participants were 349 pregnant women who were regular smokers upon learning of their pregnancy: 118 of them quit smoking prior to initiating prenatal care, and 231 were still smoking at the start of prenatal care. Smoking status for all participants was biochemically verified using urine cotinine testing. Participants were recruited from obstetrical clinics located in the greater Burlington, Vermont area. The University of Vermont College of Medicine's Institutional Review Board approved these
Results
The use of the algorithm to exclude data sets resulted in 14% (49/349) of cases being excluded (Table 1): Rule 1 eliminated 25 cases, Rule 2 eliminated 32 cases, and the use of both criteria eliminated 49 cases (i.e., 8 participants met exclusion criteria under both rules). The use of R2 excluded 16% (55/349) of cases; 45 were excluded because R2 < 0, and 10 were removed because indifference points were identical across all seven time points (flat response set) and so a R2 statistic could not be
Discussion
This study aimed to systematically examine the differential impact of two methods for characterizing the orderliness of DD data: the algorithmic method and conventional statistical model fit. DD data were systematically eliminated from a large data set of data from pregnant smokers, using various combinations of the methods, to see the effects on the data set. We found that in our data set (349 pregnant women who were smoking when they learned of pregnancy), a large percentage of subjects (14%)
Role of funding source
Nothing declared.
Contributors
Dr. White and Dr. Redner devised the concept for the paper and proposed the analyses; Dr. White wrote the manuscript. Ms. Skelly conducted the data analyses and wrote the statistical methods section. She approved the statistical language in the results section and tables, and provided feedback on the manuscript. Dr. Higgins was the lab director and principal investigator for the pregnant smokers project from which this data set derived. He was also the postdoctoral mentor to Dr. White and Dr.
Conflicts of interest
None.
Acknowledgements
This research was supported by National Institutes of Health Center of Biomedical Research Excellence award P20GM103644 from the National Institute of General Medical Sciences, Tobacco Centers of Regulatory Science award P50DA036114 from the National Institute on Drug Abuse and U.S. Food and Drug Administration, research grants R01DA14028 and R01HD075669 from the National Institute on Drug Abuse and National Institute of Child Health and Human Development, respectively, and Institutional
References (19)
- et al.
Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: emerging evidence
Pharmacol. Ther.
(2012) - et al.
Immediate pleasures and future consequences. A neuropsychological study of binge eating and obesity
Appetite
(2010) - et al.
Examining delay discounting of condom-protected sex among opioid-dependent women and non-drug-using control women
Drug Alcohol Depend.
(2014) - et al.
Toward a behavioral economic understanding of drug dependence: delay discounting processes
Addiction
(2001) The association between individual time preferences and health maintenance habits
Med. Decis. Making
(2010)- et al.
Temporal discounting: basic research and the analysis of socially important behavior
J. Appl. Behav. Anal.
(2001) - et al.
Exponential versus hyperbolic discounting of delayed outcomes: risk and waiting time
Am. Zool.
(1996) - et al.
Within-subject comparison of real and hypothetical money rewards in delay discounting
J. Exp. Anal. Behav.
(2002) - et al.
An algorithm for identifying nonsystematic delay-discounting data
Exp. Clin. Psychopharmacol.
(2008)
Cited by (6)
A review of tobacco regulatory science research on vulnerable populations
2019, Preventive MedicineCitation Excerpt :The fifth study provides novel cross-sectional data underscoring that use of high nicotine-yield cigarettes among women of reproductive age predicts tobacco dependence and smoking through pregnancy (Higgins et al., 2017d). Those five reports utilizing nationally-representative samples are complemented by studies with convenience samples (a) providing seminal data on the timing of smoking cessation during pregnancy (Heil et al., 2014), (b) using socioeconomic characteristics and innovative behavioral-economic tasks to predict smoking cessation and e-cigarette use (Chivers et al., 2016; White et al., 2014; White et al., 2015; Higgins et al., 2017e), and (c) providing new knowledge on the time-course of pregnancy-related changes in nicotine metabolism (Taghavi et al., 2018a; Taghavi et al., 2018b) and experimental evidence suggesting that those changes do not promote compensatory smoking (Bergeria et al., 2018). Another group of reports (Higgins et al., 2017b; Arger et al., 2017; Higgins et al., 2017c; Higgins et al., 2018) under the behavior, addiction, and impact analysis domains provided experimental evidence that VLNCs decrease the addiction potential of cigarette smoking among women of reproductive age, complementing results on VLNCs in other populations (Donny et al., 2015).
Effects of delay and probability combinations on discounting in humans
2016, Behavioural ProcessesCitation Excerpt :If the criteria were met, all data sets from that participant were removed from the analysis. This algorithm for eliminating nonsystematic delay discounting responses was chosen based on recent findings the algorithm excludes fewer cases and is uncorrelated with log k values (White et al., 2015). The above data exclusion criteria reduced the total number of response sets from 212 to 172.
The Relationship Between Nonsystematic Delay Discounting and Low-Quality Survey Responses in a Sample of Smokers: ROC Curve Analysis
2022, Experimental and Clinical PsychopharmacologyInteraction Design of Mobile-Based Product Display System
2022, Mathematical Problems in EngineeringHuman resource recommendation algorithm based on ensemble learning and Spark
2017, Journal of Physics: Conference Series