Concept: Null hypothesis
We have empirically assessed the distribution of published effect sizes and estimated power by analyzing 26,841 statistical records from 3,801 cognitive neuroscience and psychology papers published recently. The reported median effect size was D = 0.93 (interquartile range: 0.64-1.46) for nominally statistically significant results and D = 0.24 (0.11-0.42) for nonsignificant results. Median power to detect small, medium, and large effects was 0.12, 0.44, and 0.73, reflecting no improvement through the past half-century. This is so because sample sizes have remained small. Assuming similar true effect sizes in both disciplines, power was lower in cognitive neuroscience than in psychology. Journal impact factors negatively correlated with power. Assuming a realistic range of prior probabilities for null hypotheses, false report probability is likely to exceed 50% for the whole literature. In light of our findings, the recently reported low replication success in psychology is realistic, and worse performance may be expected for cognitive neuroscience.
Large-diameter trees dominate the structure, dynamics and function of many temperate and tropical forests. However, their attendant contributions to forest heterogeneity are rarely addressed. We established the Wind River Forest Dynamics Plot, a 25.6 ha permanent plot within which we tagged and mapped all 30,973 woody stems ≥1 cm dbh, all 1,966 snags ≥10 cm dbh, and all shrub patches ≥2 m(2). Basal area of the 26 woody species was 62.18 m(2)/ha, of which 61.60 m(2)/ha was trees and 0.58 m(2)/ha was tall shrubs. Large-diameter trees (≥100 cm dbh) comprised 1.5% of stems, 31.8% of basal area, and 17.6% of the heterogeneity of basal area, with basal area dominated by Tsuga heterophylla and Pseudotsuga menziesii. Small-diameter subpopulations of Pseudotsuga menziesii, Tsuga heterophylla and Thuja plicata, as well as all tree species combined, exhibited significant aggregation relative to the null model of complete spatial randomness (CSR) up to 9 m (P≤0.001). Patterns of large-diameter trees were either not different from CSR (Tsuga heterophylla), or exhibited slight aggregation (Pseudotsuga menziesii and Thuja plicata). Significant spatial repulsion between large-diameter and small-diameter Tsuga heterophylla suggests that large-diameter Tsuga heterophylla function as organizers of tree demography over decadal timescales through competitive interactions. Comparison among two forest dynamics plots suggests that forest structural diversity responds to intermediate-scale environmental heterogeneity and disturbances, similar to hypotheses about patterns of species richness, and richness- ecosystem function. Large mapped plots with detailed within-plot environmental spatial covariates will be required to test these hypotheses.
The existence of over- and under-represented sequence motifs in genomes provides evidence of selective evolutionary pressures on biological mechanisms such as transcription, translation, ligand-substrate binding, and host immunity. In order to accurately identify motifs and other genome-scale patterns of interest, it is essential to be able to generate accurate null models that are appropriate for the sequences under study. While many tools have been developed to create random nucleotide sequences, protein coding sequences are subject to a unique set of constraints that complicates the process of generating appropriate null models. There are currently no tools available that allow users to create random coding sequences with specified amino acid composition and GC content for the purpose of hypothesis testing. Using the principle of maximum entropy, we developed a method that generates unbiased random sequences with pre-specified amino acid and GC content, which we have developed into a python package. Our method is the simplest way to obtain maximally unbiased random sequences that are subject to GC usage and primary amino acid sequence constraints. Furthermore, this approach can easily be expanded to create unbiased random sequences that incorporate more complicated constraints such as individual nucleotide usage or even di-nucleotide frequencies. The ability to generate correctly specified null models will allow researchers to accurately identify sequence motifs which will lead to a better understanding of biological processes as well as more effective engineering of biological systems.
Social media are used as main discussion channels by millions of individuals every day. The content individuals produce in daily social-media-based micro-communications, and the emotions therein expressed, may impact the emotional states of others. A recent experiment performed on Facebook hypothesized that emotions spread online, even in absence of non-verbal cues typical of in-person interactions, and that individuals are more likely to adopt positive or negative emotions if these are over-expressed in their social network. Experiments of this type, however, raise ethical concerns, as they require massive-scale content manipulation with unknown consequences for the individuals therein involved. Here, we study the dynamics of emotional contagion using a random sample of Twitter users, whose activity (and the stimuli they were exposed to) was observed during a week of September 2014. Rather than manipulating content, we devise a null model that discounts some confounding factors (including the effect of emotional contagion). We measure the emotional valence of content the users are exposed to before posting their own tweets. We determine that on average a negative post follows an over-exposure to 4.34% more negative content than baseline, while positive posts occur after an average over-exposure to 4.50% more positive contents. We highlight the presence of a linear relationship between the average emotional valence of the stimuli users are exposed to, and that of the responses they produce. We also identify two different classes of individuals: highly and scarcely susceptible to emotional contagion. Highly susceptible users are significantly less inclined to adopt negative emotions than the scarcely susceptible ones, but equally likely to adopt positive emotions. In general, the likelihood of adopting positive emotions is much greater than that of negative emotions.
A rigorous systematic review and meta-analysis focused on randomised controlled trials (RCTs) of non-individualised homeopathic treatment has not previously been reported. We tested the null hypothesis that the main outcome of treatment using a non-individualised (standardised) homeopathic medicine is indistinguishable from that of placebo. An additional aim was to quantify any condition-specific effects of non-individualised homeopathic treatment.
- Proceedings of the National Academy of Sciences of the United States of America
- Published almost 3 years ago
Coastal hypoxia (dissolved oxygen ≤ 2 mg/L) is a growing problem worldwide that threatens marine ecosystem services, but little is known about economic effects on fisheries. Here, we provide evidence that hypoxia causes economic impacts on a major fishery. Ecological studies of hypoxia and marine fauna suggest multiple mechanisms through which hypoxia can skew a population’s size distribution toward smaller individuals. These mechanisms produce sharp predictions about changes in seafood markets. Hypoxia is hypothesized to decrease the quantity of large shrimp relative to small shrimp and increase the price of large shrimp relative to small shrimp. We test these hypotheses using time series of size-based prices. Naive quantity-based models using treatment/control comparisons in hypoxic and nonhypoxic areas produce null results, but we find strong evidence of the hypothesized effects in the relative prices: Hypoxia increases the relative price of large shrimp compared with small shrimp. The effects of fuel prices provide supporting evidence. Empirical models of fishing effort and bioeconomic simulations explain why quantifying effects of hypoxia on fisheries using quantity data has been inconclusive. Specifically, spatial-dynamic feedbacks across the natural system (the fish stock) and human system (the mobile fishing fleet) confound “treated” and “control” areas. Consequently, analyses of price data, which rely on a market counterfactual, are able to reveal effects of the ecological disturbance that are obscured in quantity data. Our results are an important step toward quantifying the economic value of reduced upstream nutrient loading in the Mississippi Basin and are broadly applicable to other coupled human-natural systems.
Misinterpretation and abuse of statistical tests, confidence intervals, and statistical power have been decried for decades, yet remain rampant. A key problem is that there are no interpretations of these concepts that are at once simple, intuitive, correct, and foolproof. Instead, correct use and interpretation of these statistics requires an attention to detail which seems to tax the patience of working scientists. This high cognitive demand has led to an epidemic of shortcut definitions and interpretations that are simply wrong, sometimes disastrously so-and yet these misinterpretations dominate much of the scientific literature. In light of this problem, we provide definitions and a discussion of basic statistics that are more general and critical than typically found in traditional introductory expositions. Our goal is to provide a resource for instructors, researchers, and consumers of statistics whose knowledge of statistical theory and technique may be limited but who wish to avoid and spot misinterpretations. We emphasize how violation of often unstated analysis protocols (such as selecting analyses for presentation based on the P values they produce) can lead to small P values even if the declared test hypothesis is correct, and can lead to large P values even if that hypothesis is incorrect. We then provide an explanatory list of 25 misinterpretations of P values, confidence intervals, and power. We conclude with guidelines for improving statistical interpretation and reporting.
The microbiota of the aged is variously described as being more or less diverse than that of younger cohorts, but the comparison groups used and the definitions of the aged population differ between experiments. The differences are often described by null hypothesis statistical tests, which are notoriously irreproducible when dealing with large multivariate samples. We collected and examined the gut microbiota of a cross-sectional cohort of more than 1,000 very healthy Chinese individuals who spanned ages from 3 to over 100 years. The analysis of 16S rRNA gene sequencing results used a compositional data analysis paradigm coupled with measures of effect size, where ordination, differential abundance, and correlation can be explored and analyzed in a unified and reproducible framework. Our analysis showed several surprising results compared to other cohorts. First, the overall microbiota composition of the healthy aged group was similar to that of people decades younger. Second, the major differences between groups in the gut microbiota profiles were found before age 20. Third, the gut microbiota differed little between individuals from the ages of 30 to >100. Fourth, the gut microbiota of males appeared to be more variable than that of females. Taken together, the present findings suggest that the microbiota of the healthy aged in this cross-sectional study differ little from that of the healthy young in the same population, although the minor variations that do exist depend upon the comparison cohort. IMPORTANCE We report the large-scale use of compositional data analysis to establish a baseline microbiota composition in an extremely healthy cohort of the Chinese population. This baseline will serve for comparison for future cohorts with chronic or acute disease. In addition to the expected difference in the microbiota of children and adults, we found that the microbiota of the elderly in this population was similar in almost all respects to that of healthy people in the same population who are scores of years younger. We speculate that this similarity is a consequence of an active healthy lifestyle and diet, although cause and effect cannot be ascribed in this (or any other) cross-sectional design. One surprising result was that the gut microbiota of persons in their 20s was distinct from those of other age cohorts, and this result was replicated, suggesting that it is a reproducible finding and distinct from those of other populations.
Can behavior be unconsciously primed via the activation of attitudes, stereotypes, or other concepts? A number of studies have suggested that such priming effects can occur, and a prominent illustration is the claim that individuals' accuracy in answering general knowledge questions can be influenced by activating intelligence-related concepts such as professor or soccer hooligan. In 9 experiments with 475 participants we employed the procedures used in these studies, as well as a number of variants of those procedures, in an attempt to obtain this intelligence priming effect. None of the experiments obtained the effect, although financial incentives did boost performance. A Bayesian analysis reveals considerable evidential support for the null hypothesis. The results conform to the pattern typically obtained in word priming experiments in which priming is very narrow in its generalization and unconscious (subliminal) influences, if they occur at all, are extremely short-lived. We encourage others to explore the circumstances in which this phenomenon might be obtained.
Archaeological data provide alternative hypotheses on Pacific herring (Clupea pallasii) distribution, abundance, and variability
- Proceedings of the National Academy of Sciences of the United States of America
- Published almost 6 years ago
Pacific herring (Clupea pallasii), a foundation of coastal social-ecological systems, is in decline throughout much of its range. We assembled data on fish bones from 171 archaeological sites from Alaska, British Columbia, and Washington to provide proxy measures of past herring distribution and abundance. The dataset represents 435,777 fish bones, dating throughout the Holocene, but primarily to the last 2,500 y. Herring is the single-most ubiquitous fish taxon (99% ubiquity) and among the two most abundant taxa in 80% of individual assemblages. Herring bones are archaeologically abundant in all regions, but are superabundant in the northern Salish Sea and southwestern Vancouver Island areas. Analyses of temporal variability in 50 well-sampled sites reveals that herring exhibits consistently high abundance (>20% of fish bones) and consistently low variance (<10%) within the majority of sites (88% and 96%, respectively). We pose three alternative hypotheses to account for the disjunction between modern and archaeological herring populations. We reject the first hypothesis that the archaeological data overestimate past abundance and underestimate past variability. We are unable to distinguish between the second two hypotheses, which both assert that the archaeological data reflect a higher mean abundance of herring in the past, but differ in whether variability was similar to or less than that observed recently. In either case, sufficient herring was consistently available to meet the needs of harvesters, even if variability is damped in the archaeological record. These results provide baseline information prior to herring depletion and can inform modern management.