How To Read Medical Journal Articles: Just Because It’s On Paper Doesn’t Mean It’s Necessarily True

For the past few days I have been struggling with writing an entry about two articles (and 2 accompanying editorials) on diabetes complications published in last week’s New England Journal of Medicine. My struggle has been how to convey my thoughts about these two very important articles in a coherent yet concise manner. I must confess that my webmaster has scolded me on several occasions for rambling on and on in some of my entries. He has told me that most readers have a limited attention span and that “word bites” work better than epics. I respect his opinion about such matters but I don’t agree completely; I would like to think that my readers want to go beyond superficial discussions yet not get bogged down in unimportant details. Anyway, as I thought about these matters, it dawned on me that many readers may not have a clue how to read medical journal articles. By “read” I mean how to analyze whether the authors’ claims are supported by the data presented and whether the study design makes sense.

I am not suggesting that all my readers should be expert readers of medical research papers, but rather, that one should understand how it can be that firm conclusions reached in medical studies based on analysis of the data are often completely wrong (we’re not talking about data that has been falsified).

An example of good data yet incorrect conclusions

Let me give you an example . In 1973, the New England Journal of Medicine published a study that showed increased levels of hemoglobin A1c (HbA1c) in the blood of people with diabetes; HbA1c is a minor hemoglobin component that represents about 5% of the total hemoglobin (I assume you already know that hemoglobin is the blood protein that carries oxygen around). In people with diabetes, the level of HbA1c was found to be about 10%, confirming the results of a small study carried out a few years earlier. The investigators were not able to demonstrate any relationship between the level of HbA1c and anything they could measure- blood glucose levels, diabetes complications, etc. They concluded that HbA1c was possibly a genetic marker for diabetes. Of course, many years later we all know that HbA1c is a very valuable measure of average blood glucose and a risk factor for diabetes complications. So here we have reliable data on hemoglobin A1c but faulty conclusions because the assessments for blood glucose and diabetes complications were not sensitive enough to uncover what HbA1c was really all about.

Analyzing the structure of medical journal scientific articles

Of course, the first thing one sees in looking at a medical report is the title and a list of the authors including each author’s place of work, typically a university, scientific institute, or company. Next there is the “summary,” which is, as the name implies, a brief discussion of the study goals, methods, results, and conclusions. The summary is useful for scanning medical journal articles to identify those that might be worthwhile to read. A word of caution- one shouldn’t assume that the summary conclusion has any validity.

Next, there is the “introduction,” which ideally provides background information about the study subject and the specific study goals. Basically, this section provides the rationale for the study and what the investigators were hoping to learn.

Next comes the “methods” section which describes in some detail how the study was designed and what statistical methods were used to analyze the data. Thus, a study comparing the effects of 2 different antibiotics on strep throat would provide detailed information on the study population, including how the subjects were selected (e.g, age range), how strep throat was diagnosed , and what things were assessed to carry out the comparison (e.g., number of days of fever after starting the antibiotic). In many studies, the methods section contains information that is crucial in determining if the study conclusions are valid given the study results. For example, if the study population included only people over age 40 years, it would be risky to generalize the conclusions to people of all ages.

The next section is “results.” Here, the authors present the data, unadorned by opinion. The data might include how many patients were compliant with their treatment regimen, what side-effects were found, etc. Well-written studies present the results in a very organized way and take advantage of tables and figures if they make it easier for the reader to digest the material. This section includes the results of any statistical analyses carried out (i.e, were there statistically significant differences in outcomes comparing the 2 antibiotics and how significant were the differences, if any?).

The last section is the “discussion.” This is where the authors get to interpret the study results and convey their opinions regarding the significance (meaning the importance of the results as opposed to the statistical aspects) of the results. For example, they may conclude based on the results that antibiotic B was far superior to antibiotic A and with fewer side-effects, and should become the standard of care for strep throat. In this section, authors are supposed to address what the limitations of the study might have been. For example, if the study showed no statistically significant differences between the 2 antibiotics, maybe there might have been differences if the antibiotics doses had been different?

It is the job of the reader to determine if the conclusions expressed in the discussion section accurately reflect the data in the results section. Also, the reader needs to consider if statistically significant differences found in a study are also clinically significant. For example, it might be that antibiotic B decreased the duration of fever on average from 4 days to 3.8 days. The different might be statistically significant but not clinically important (particularly if antibiotic B had more side effects or cost 3 times as much as antibiotic A).

Summary

Now are you are ready to tackle the medical journals with authority? I hope I conveyed just how complicated it can be to evaluate medical claims, even for highly sophisticated folk. I think the most important thing is to determine if the study conclusions are appropriate given the results. You may be surprised to find that often study conclusions are either much more speculative than the data warrant or that serious limitations of the study results have not been addressed.

• ← More About Congenital Hypothyroidism
• The Relationship Between Glycemic Control and Cardiovascular Outcomes in Type 2 Diabetes: Results of the ACCORD and ADVANCE Studies →