by Elizabeth Hofheinz, M.P.H., M.Ed., February 27, 2020
While box plots and whiskers seem a world away from internal fixation and reaming, they are indeed related.
Whether in spine, hip or shoulder, it would be difficult to argue with the notion that many of the articles appearing in the orthopedic literature have vastly improved how surgeons work and how patients live their lives.
And yet, there is a gaping hole in the education of orthopedic residents when it comes to comprehending the statistics behind the literature.
To determine the extent of this lack, a team of researchers from the University of Alabama at Birmingham and the University of Arizona set out to assess the extent to which orthopaedic residents know about and can apply biostatistics and study design concepts.
Brett Ponce, M.D. is a professor at the University of Alabama and was a co-author on the paper, “A National Survey of Orthopaedic Residents Identifies Deficiencies in the Understanding of Medical Statistics,” which appears in the December 31, 2019 edition of The Journal of Bone and Joint Surgery.
Dr. Ponce told OSN, “While statistics are peppered throughout orthopedic training, they are not addressed in any organized fashion. In a typical scenario, a Ph.D.-level statistician is brought in to deliver a lecture. But then that person leaves and thus the students do not learn to apply the information. This is in contrast to surgical training where we have a lecturer and then have a chance to see the condition in the clinic and treat it in the OR.”
Using a variation of a test originally designed for internal medicine residents, the researchers obtained data from 178 orthopedic residents. Dr. Ponce: “There were 10 participating residency programs that varied in size, with the smallest program size being 14 residents and the largest having 60.”
“We looked at demographics and attitudes toward biostatistics. We wanted to determine if the residents recognized the importance of statistics. Fortunately, we found that not only did they acknowledge the importance of statistics, but they expressed a desire to learn more. There is a good chance that we are not giving residents the opportunity or time to learn statistics. Or, we are not lowering the threshold for them to learn, i.e., requiring yearly lectures on statistics instead of exposing them infrequently.”
“It also doesn’t help that statistics are not stressed on the board exams,” said Dr. Ponce. “We have questions on the boards that are irrelevant to the vast majority of our practices—such as histology. Such questions are relevant to orthopaedic oncologists, who make up less than 1% of all orthopaedic surgeons. But statistics and how to read the medical literature—things that are relevant across all subspecialties—are not appropriately weighted in either the volume or type of questions asked.”
Overall, says Dr. Ponce, “residents in larger training programs were more knowledgeable about statistics than those in smaller programs, in part because larger programs may have a statistician who can give lectures on a routine basis or greater resources for research which may help create a culture of prioritizing statistics.”
First, do no harm…
“Medical schools should start incorporating a greater number of statistics questions on the United States Medical Licensing Examination; there should also be more biostatistical testing on residency in-training exams, accreditation and recertification exams. And ideally, residency programs would have standardized statistics lectures so that smaller programs are not disadvantaged.”
“These standardized exams typically contain questions on such real-world topics as sexual harassment and discrimination, but there are few or none on statistics. This is alarming because in some reports up to 90% of peer-reviewed articles have some type of erroneous interpretation or application of data analysis. How can we do no harm if we don’t know what we are doing?”
Change in the air?
Asked if there were a movement afoot to change things, Dr. Ponce noted, “To my knowledge, no one is spearheading this initiative. There are several potential reasons for this. Physicians in general, including orthopaedic surgeons, may not be comfortable with statistics and therefore may shy away from teaching or stressing the importance of statistics. As statistics are not particular to orthopaedic surgery, there may be a perception that a histology slide on a rare bone tumor is more applicable to our practice than appreciating the importance of using the proper statistical test in a paper.”
“It’s hard to imagine who would be a passionate advocate for promoting education in statistics when there is minimal support for educational activities that are not industry, implant or technique driven. Surgeons are paid to treat and operate, not be experts in statistics. Additionally, there is a lack of standardization across medical journals. If some journals have a statistics review by a trained statistician and other journals may not have the same vetting statistician standards, then where does that leave us as a field?”
“For example, take a grade 3 acromioclavicular joint separation. In several systematic reviews, the results of surgery are typically the same as those of nonoperative treatment. While there may be extenuating circumstances supporting acute surgical intervention, as soon as you operate you open up the potential for surgical complications. While the majority of patients with this type of injury receive nonoperative treatment, there can be surgeons who cite an underpowered paper saying that these patients do well with surgical treatment as justification to intervene operatively. Such an approach to the literature is in keeping with what a mentor from my residency once said, “Physicians tend to use the literature like a drunk uses a lamppost – more for support than for enlightenment.”
Dr. Ponce and his colleagues hope that their study raises awareness about this yawning hole in orthopedic education. “Over time I hope this and future related work will alter clinical practice. We took an oath to do no harm and we must adhere to that by properly applying the statistics we are reading in papers.”
Alas, how else will surgeons grow? A few tips on statistics during journal club is not going to cut it.
