Clarity and accuracy of reporting are fundamental to the scientific process. Readability formulas can estimate how difficult a text is to read. Here, in a corpus consisting of 709,577 abstracts published between 1881 and 2015 from 123 scientific journals, we show that the readability of science is steadily decreasing. Our analyses show that this trend is indicative of a growing use of general scientific jargon. These results are concerning for scientists and for the wider public, as they impact both the reproducibility and accessibility of research findings.
- Proceedings of the National Academy of Sciences of the United States of America
- Published over 6 years ago
Recent advances in Bayesian hypothesis testing have led to the development of uniformly most powerful Bayesian tests, which represent an objective, default class of Bayesian hypothesis tests that have the same rejection regions as classical significance tests. Based on the correspondence between these two classes of tests, it is possible to equate the size of classical hypothesis tests with evidence thresholds in Bayesian tests, and to equate P values with Bayes factors. An examination of these connections suggest that recent concerns over the lack of reproducibility of scientific studies can be attributed largely to the conduct of significance tests at unjustifiably high levels of significance. To correct this problem, evidence thresholds required for the declaration of a significant finding should be increased to 25-50:1, and to 100-200:1 for the declaration of a highly significant finding. In terms of classical hypothesis tests, these evidence standards mandate the conduct of tests at the 0.005 or 0.001 level of significance.
Biologists should submit their preprints to open servers, a practice common in mathematics and physics, to open and accelerate the scientific process.
There is a growing movement to encourage reproducibility and transparency practices in the scientific community, including public access to raw data and protocols, the conduct of replication studies, systematic integration of evidence in systematic reviews, and the documentation of funding and potential conflicts of interest. In this survey, we assessed the current status of reproducibility and transparency addressing these indicators in a random sample of 441 biomedical journal articles published in 2000-2014. Only one study provided a full protocol and none made all raw data directly available. Replication studies were rare (n = 4), and only 16 studies had their data included in a subsequent systematic review or meta-analysis. The majority of studies did not mention anything about funding or conflicts of interest. The percentage of articles with no statement of conflict decreased substantially between 2000 and 2014 (94.4% in 2000 to 34.6% in 2014); the percentage of articles reporting statements of conflicts (0% in 2000, 15.4% in 2014) or no conflicts (5.6% in 2000, 50.0% in 2014) increased. Articles published in journals in the clinical medicine category versus other fields were almost twice as likely to not include any information on funding and to have private funding. This study provides baseline data to compare future progress in improving these indicators in the scientific literature.
Science graduates require critical thinking skills to deal with the complex problems they will face in their 21st century workplaces. Inquiry-based curricula can provide students with the opportunities to develop such critical thinking skills; however, evidence suggests that an inappropriate level of autonomy provided to underprepared students may not only be daunting to students but also detrimental to their learning. After a major review of the Bachelor of Science, we developed, implemented, and evaluated a series of three vertically integrated courses with inquiry-style laboratory practicals for early-stage undergraduate students in biomedical science. These practical curricula were designed so that students would work with increasing autonomy and ownership of their research projects to develop increasingly advanced scientific thinking and communication skills. Students undertaking the first iteration of these three vertically integrated courses reported learning gains in course content as well as skills in scientific writing, hypothesis construction, experimental design, data analysis, and interpreting results. Students also demonstrated increasing skills in both hypothesis formulation and communication of findings as a result of participating in the inquiry-based curricula and completing the associated practical assessment tasks. Here, we report the specific aspects of the curricula that students reported as having the greatest impact on their learning and the particular elements of hypothesis formulation and communication of findings that were more challenging for students to master. These findings provide important implications for science educators concerned with designing curricula to promote scientific thinking and communication skills alongside content acquisition.
There is a replication crisis spreading through the annals of scientific inquiry. Although some work has been carried out to uncover the roots of this issue, much remains unanswered. With this in mind, this paper investigates how the gender of the experimenter may affect experimental findings. Clinical trials are regularly carried out without any report of the experimenter’s gender and with dubious knowledge of its influence. Consequently, significant biases caused by the experimenter’s gender may lead researchers to conclude that therapeutics or other interventions are either overtreating or undertreating a variety of conditions. Bearing this in mind, this policy paper emphasizes the importance of reporting and controlling for experimenter gender in future research. As backdrop, it explores what we know about the role of experimenter gender in influencing laboratory results, suggests possible mechanisms, and suggests future areas of inquiry.
Causal inference is a core task of science. However, authors and editors often refrain from explicitly acknowledging the causal goal of research projects; they refer to causal effect estimates as associational estimates. This commentary argues that using the term “causal” is necessary to improve the quality of observational research. Specifically, being explicit about the causal objective of a study reduces ambiguity in the scientific question, errors in the data analysis, and excesses in the interpretation of the results. (Am J Public Health. Published online ahead of print March 22, 2018: e1-e4. doi:10.2105/AJPH.2018.304337).
Scientists are under increasing pressure to do “novel” research. Here I explore whether there are risks to overemphasizing novelty when deciding what constitutes good science. I review studies from the philosophy of science to help understand how important an explicit emphasis on novelty might be for scientific progress. I also review studies from the sociology of science to anticipate how emphasizing novelty might impact the structure and function of the scientific community. I conclude that placing too much value on novelty could have counterproductive effects on both the rate of progress in science and the organization of the scientific community. I finish by recommending that our current emphasis on novelty be replaced by a renewed emphasis on predictive power as a characteristic of good science.
- Proceedings of the National Academy of Sciences of the United States of America
- Published over 4 years ago
Concerns about a lack of reproducibility of statistically significant results have recently been raised in many fields, and it has been argued that this lack comes at substantial economic costs. We here report the results from prediction markets set up to quantify the reproducibility of 44 studies published in prominent psychology journals and replicated in the Reproducibility Project: Psychology. The prediction markets predict the outcomes of the replications well and outperform a survey of market participants' individual forecasts. This shows that prediction markets are a promising tool for assessing the reproducibility of published scientific results. The prediction markets also allow us to estimate probabilities for the hypotheses being true at different testing stages, which provides valuable information regarding the temporal dynamics of scientific discovery. We find that the hypotheses being tested in psychology typically have low prior probabilities of being true (median, 9%) and that a “statistically significant” finding needs to be confirmed in a well-powered replication to have a high probability of being true. We argue that prediction markets could be used to obtain speedy information about reproducibility at low cost and could potentially even be used to determine which studies to replicate to optimally allocate limited resources into replications.
The gender imbalance in STEM subjects dominates current debates about women’s underrepresentation in academia. However, women are well represented at the Ph.D. level in some sciences and poorly represented in some humanities (e.g., in 2011, 54% of U.S. Ph.D.’s in molecular biology were women versus only 31% in philosophy). We hypothesize that, across the academic spectrum, women are underrepresented in fields whose practitioners believe that raw, innate talent is the main requirement for success, because women are stereotyped as not possessing such talent. This hypothesis extends to African Americans' underrepresentation as well, as this group is subject to similar stereotypes. Results from a nationwide survey of academics support our hypothesis (termed the field-specific ability beliefs hypothesis) over three competing hypotheses.