Phrases such as “trending toward significance,” “approaching statistical significance,” “almost statistically significant,” or “on the brink of significance” should never be used to describe a P value. And yet you will see this every now and again, even in credible peer-reviewed journals.
The P value is used to provide a standard point at which research outcomes are considered to be by design, not by chance. Often the implication being made when a phrase like “trending toward significance” is employed is that, with additional data, the study would have reached significance. Additional data means a larger sample size, but sample size is not a random number, nor is it the largest enrolment a study can achieve. Researchers calculate sample size to determine the number of enrolled patients/samples needed to provide a robust, reliable, and reproducible outcome. While the challenges of enrolment in clinical trials highlight the problem of having too few patients enrolled in a study to meet the sample size required for adequate statistical analysis, exceeding the sample size can also skew the results of a statistical analysis powered for fewer samples.
As well, P<.05 is a generous standard. (As Wood and colleagues conclude, “P values in the region of 0.05 represent quite modest degrees of evidence, whichever side of the divide they lie on.”) P values close to .05 may be “interesting hints,” not solid outcomes on which conclusions should be based. Suggesting near significance is, in the words of Wood and his coauthors again, “not just inappropriate, but actively misleading.”
An Easy Solution
First, understand the ethical ramifications of implying significance where there is none. Given that the ultimate goal of medical research is translation into clinical care, either indirectly (eg, a basic research study providing the basis for future studies) or directly (eg, a clinical trial providing the data for FDA approval of a drug), implying significance in the absence of evidence supporting that conclusion can lead to the waste of time and other resources, unneeded clinical testing, and even the translation of an ineffective or dangerous treatment into clinical care.
Second, apply an easy solution to the problem: In addition to banning the phrases “trending toward significance,” “approaching statistical significance,” “almost statistically significant,” or “on the brink of significance” when writing or editing, follow best practices for reporting research results. Doing so will help research teams, writers, and editors actively avoid misrepresenting outcomes. Briefly, best practice in reporting statistics dictates that P values should never be reported without the data being compared (ie, a rate or absolute number) and appropriate measures of uncertainty. The P value provides information about the statistical testing of the hypothesis—is the null hypothesis true or not?—while the data being compared and measures of uncertainty provide the reader with important quantitative information.
Want to learn more?
Check out this BMJ article that provides the statistician’s explanation: Wood J, Freemantle N, King M, Nazareth I. Trap of trends to statistical significance: likelihood of near significant P value becoming more significant with extra data. BMJ. 2014;348:g2215. doi:10.1136/bmj.g2215
The AMA Manual of Style, 11/e provides a great starting point for new writers and editors, and How to Report Statistics in Medicine, 2/e by Thomas A. Lang and Michelle Secic delves more deeply into best practices for reporting statistics that every medical writer and editor should know.
Thanks for bringing up these points.
Thank you, Barbara! We are pleased you like the piece.