Concept: Null result
We explore whether the number of null results in large National Heart Lung, and Blood Institute (NHLBI) funded trials has increased over time.
Evaluations of service delivery interventions with contemporaneous controls often yield null results, even when the intervention appeared promising in advance. There can be many reasons for null results. In this paper we introduce the concept of a ‘rising tide’ phenomenon being a possible explanation of null results. We note that evaluations of service delivery interventions often occur when awareness of the problems they intend to address is already heightened, and pressure to tackle them is mounting throughout a health system. An evaluation may therefore take place in a setting where the system as a whole is improving - where there is a pronounced temporal trend or a ‘rising tide causing all vessels to rise’. As a consequence, control sites in an intervention study will improve. This reduces the difference between intervention and control sites and predisposes the study to a null result, leading to the conclusion that the intervention has no effect. We discuss how a rising tide may be distinguished from other causes of improvement in both control and intervention groups, and give examples where the rising tide provides a convincing explanation of such a finding. We offer recommendations for interpretation of research findings where improvements in the intervention group are matched by improvements in the control group. Understanding the rising tide phenomenon is important for a more nuanced interpretation of null results arising in the context of system-wide improvement. Recognition that a rising tide may have predisposed to a null result in one health system cautions against generalising the result to another health system where strong secular trends are absent.
There is preliminary evidence that approach avoid training can shift implicit alcohol associations and improve treatment outcomes. We sought to replicate and extend those findings in US undergraduate social drinkers (Study 1) and at-risk drinkers (Study 2). Three adaptations of the approach avoid task (AAT) were tested. The first adaptation - the approach avoid training - was a replication and targeted implicit alcohol approach associations. The remaining two adaptations - the general identity and personalized identity trainings - targeted implicit drinking identity associations, which are robust predictors of hazardous drinking in US undergraduates. Study 1 included 300 undergraduate social drinkers. They were randomly assigned to real or sham training conditions for one of the three training adaptations, and completed two training sessions, spaced one week apart. Study 2 included 288 undergraduates at risk for alcohol use disorders. The same training procedures were used, but the two training sessions occurred within a single week. Results were not as expected. Across both studies, the approach avoid training yielded no evidence of training effects on implicit alcohol associations or alcohol outcomes. The general identity training also yielded no evidence of training effects on implicit alcohol associations or alcohol outcomes with one exception; individuals who completed real training demonstrated no changes in drinking refusal self-efficacy whereas individuals who completed sham training had reductions in self-efficacy. Finally, across both studies, the personalized identity training yielded no evidence of training effects on implicit alcohol associations or alcohol outcomes. Despite having relatively large samples and using a well-validated training task, study results indicated all three training adaptations were ineffective at this dose in US undergraduates. These findings are important because training studies are costly and labor-intensive. Future research may benefit from focusing on more severe populations, pairing training with other interventions, increasing training dose, and increasing gamification of training tasks.
The behavioral urgency hypothesis suggests that stimuli signaling potential danger will receive attentional priority. However, results from the gaze cueing paradigm have failed to consistently show that emotional expression modulates gaze following. One possible explanation for these null results is that participants are repeatedly exposed to the same emotional expressions during the typical gaze cueing procedure. We employed a relatively novel gaze cueing method in which participants were presented with 2 unique (or “rare”) trials during an experimental block. Specifically, either 2 fearful face trials appeared within a block of happy faces or 2 happy face trials appeared within a block of fearful faces. Results showed that when participants were repeatedly exposed to the same emotional expression, gaze cueing was independent of face type. However, when the emotional expression was a rare event, significantly larger cueing occurred for fearful than for happy faces. These results support the behavioral urgency hypothesis and show that emotional expression does indeed modulate gaze following. (PsycINFO Database Record
In two experiments, Brookshire, Ivry, and Casasanto (2010) showed that words with positive and negative emotional valence can activate spatial representations with a high degree of automaticity, but also that this activation is highly context dependent. Lebois, Wilson-Mendenhall, and Barsalou (2015) reported that they “aimed to replicate” our study but found only null results in the “Brookshire et al. replication” conditions. Here we express concerns about three aspects of this paper. First, the study was not an attempt to replicate ours; it was a different study that adapted our method. Second, Lebois et al. did not accurately represent our theoretical position. Third, Lebois et al.’s main conclusion, that spatial congruity effects depend on the task context, was not supported by their data. Despite these concerns, we agree with Lebois et al.’s overall message that spatial aspects of words' meanings are activated differently in different contexts. This was a main conclusion of our study as well.
A Framework for Testing and Promoting Expanded Dissemination of Promising Preventive Interventions That Are Being Implemented in Community Settings
- Prevention science : the official journal of the Society for Prevention Research
- Published about 4 years ago
Many evidence-based preventive interventions have been developed in recent years, but few are widely used. With the current focus on efficacy trials, widespread dissemination and implementation of evidence-based interventions are often afterthoughts. One potential strategy for reversing this trend is to find a promising program with a strong delivery vehicle in place and improve and test the program’s efficacy through rigorous evaluation. If the program is supported by evidence, the dissemination vehicle is already in place and potentially can be expanded. This strategy has been used infrequently and has met with limited success to date, in part, because the field lacks a framework for guiding such research. To address this gap, we outline a framework for moving promising preventive interventions that are currently being implemented in community settings through a process of rigorous testing and, if needed, program modification in order to promote expanded dissemination. The framework is guided by RE-AIM (Reach, Efficacy/Effectiveness, Adoption, Implementation, and Maintenance) (Glasgow et al., Am J Publ Health 89:1322-1327, 1999), which focuses attention on external as well as internal validity in program tests, and is illustrated with examples. Challenges, such as responding to negative and null results, and opportunities inherent in the framework are discussed.
There is a rapidly increasing trend in media-media multitasking or MMM (using two or more media concurrently). In a recent conference, scholars from diverse disciplines expressed concerns that indulgence in MMM may compromise well-being and/or cognitive abilities. However, research on MMM’s impacts is too sparse to inform the general public and policy makers whether MMM should be encouraged, managed, or minimized. The primary purpose of the present study was to develop an innovative computerized instrument - the Survey of the Previous Day (SPD) - to quantify MMM as well as media-nonmedia and nonmedia-nonmedia multitasking and sole-tasking. The secondary purpose was to examine whether these indices could predict a sample of well-being related, psychosocial measures. In the SPD, participants first recalled (typed) what they did during each hour of the previous day. In later parts of the SPD, participants analysed activities and their timing and duration for each hour of the previous day, while relevant recall was on display. Participants also completed the Media Use Questionnaire. The results showed non-significant relationship between tasking measures and well-being related measures. Given how little is known about the associations between MMM and well-being, the null results may offer some general reassurance to those who are apprehensive about negative impacts of MMM.
We would like to thank Schafhalter-Zoppoth et al. for highlighting the growing numbers of Latinos in prison, who have an increased vulnerability to poor health outcomes upon release. We agree that by excluding Spanish-speaking Latinos, we are unable to generalize our results to this population. But we disagree that their exclusion would bias our study to null results. (Am J Public Health. Published online ahead of print April 18, 2013: e1. doi:10.2105/AJPH.2013.301363).