By: Catherine Ebeling, RN, BSN
Every day we read headlines like, “high fat diet can cause heart disease”, “consumption of whole grain products decreases risk of cancer by 36%” “red meat linked to colon cancer”.
Sometimes one headline actually contradicts the last headline.
Did you ever stop to wonder if these headlines are true? Just because it appears on the front page of a news article or you heard it from your physician does not mean it is true or accurate.
Do you know the difference between good studies and bad studies? Many of the so-called ‘studies’ we read about have absolutely no basis in fact.
What you need to know is that there is a huge discrepancy in how studies are conducted and what the data means; how the message is interpreted, and how that message is delivered to you.
When I was in nursing school at a major medical and research university, I was told that only a very small percentage of scientific studies (less than 10%) actually made it to mainstream media and were published.
And of the ones we do hear about? Many of those are flat out wrong, misinterpreted, biased or totally skewed.
How does this happen? It’s science, after all.
Let’s take a look at how our brains work. Since the days of cavemen, the human brain has been hardwired to look for patterns. ‘A causes B’ to happen. It was integral to our very survival.
However, as time went on, many of those ‘A causes B to happen’ were purely coincidental. A long drought occurs and dehydrated, desperate natives begin to dance around, and lo and behold, it rains. The ‘rain dance’ is born. Cause and effect? Coincidence.
Regardless, our brains want to make a connection. And we tend to generate our beliefs consistent to what we already believe.
Along comes the scientific method. Now many of us may or may not remember this from school, but scientific method works like this:
- - An observation is made
- - A hypothesis is formed as a possible explanation
- - Conduct experiments
- - Collect data
- - Reach a conclusion
- - The results must be CONSISTENT and REPEATABLE
Keep this in mind as I go on...
‘Studies’ that we read about in the news can be one of two things:
1. Observation studies - An observation of a particular situation, and some sort of conclusion is drawn based on that observation. There is no intervention or treatment, just observations and gathering of data.
2. A true clinical or scientific study is conducted as in the six steps above.
In both of these, we look for traits, behaviors or results that are associated, correlated, or linked. As in ‘A causes B’, or ‘smoking is associated with lung cancer’.
The problem though, with purely observation studies is that ‘A’ does not always cause ‘B’ to happen. These observations are notoriously inaccurate. Many other factors may be involved which affect the outcome.
These other variables are called ‘comfounding’ variables. As in this example: It was thought that reading to your child helps him/her become a good student and get better grades in school. ‘A causes B’, right?
The conclusion was made that reading caused your child to become a better student. But consider this - parents who read to their children are generally more intelligent and better educated, so their children get better grades in school anyway. Enter another variable.
An observational study can only make observations and form a loose hypothesis (untested) on that observation. Did you realize that most of the dietary and health headlines we read in the paper are based on observational studies?
Clinical studies are the gold standard for forming any real scientific conclusions.
Example of an observation study versus a clinical study:
The Harvard nurses study was an observational study of middle-aged nurses taking estrogen. It was observed that these nurses taking estrogen had 40% less heart disease.
Every Doctor wanted to jump on the estrogen bandwagon and start prescribing estrogen!
A clinical (remember-this is the real scientific study) study on estrogen was conducted. 16,000 women were studied for 5 years, and it was found that the women taking estrogen had a 30% HIGHER risk of heart disease.
So why did an observational study show the opposite effect?
Comfounding variables, that’s how.
As a whole, the nurses in the first, observational study were more health conscious than the general public and so had less heart disease in SPITE of taking estrogen, but the actual clinical study showed that estrogen was actually linked to more heart disease.
Unfortunately most of what we see in the news is based on observational studies. As we now know - highly inaccurate.
Scientists also skew data and interpretations differently depending on the outcome THEY want. Even in real scientific, clinical studies.
In a November 2010 article from the Atlantic Monthly, John Loannidis, one of the world’s foremost experts on the credibility of medical research,
“has shown again and again in many different ways that much of what biomedical researchers conclude in published studies - conclusions that doctors keep in mind when prescribing antibiotics or blood-pressure medication, or when they advise us to consume more fiber or less meat, or when they recommend surgery for heart disease or back pain - is misleading, exaggerated, and often flat-out wrong.”
He charges that as much as 90% of the published medical information that doctors rely on (to pass onto us) is flawed.
“…when it comes to cancer, heart disease, and other common ailments, there is plenty of published research but much of it is remarkably unscientific based largely on observation.”
And he says, 20-25% of conclusions from actual scientific, clinical studies are wrong as well.
“Researchers frequently manipulate data analysis, chase career advancing findings, over good science and....use peer review to suppress opposing views. Most often scientific research is manipulated with a particular outcome in mind BEFORE the study is conducted.”
Researchers often get involved in studies already looking for certain results—and, lo and behold, they get them!
While we tend to think of the scientific process as objective, rigorous, and unbiased, it’s very easy to manipulate results - unintentionally or even unconsciously.
Ok, so what do we do when we read about the latest ‘scientific’ study? How do you know if it it true or not?
Ask these critical thinking questions:
Is it an observational study or an actual scientific clinical study?
Remember, an observational study is a POSSIBLE connection between two things but may have other factors that cause the outcome. And it is PURELY an observation. If A and B are linked could it actually be because of C? Or C, D and E?
Does A cause B, or did B cause A? What is the cause and what is the effect?
Take this observation: Running makes you thinner. Does that mean then that basketball will make you taller? Or, could it be that good runners are thinner to begin with, much like good basketball players generally are taller than average?
Or try this one: Low carb diets make people fat. Really? Or is it because overweight people are primarily the type of people to be on a low carb diet to lose weight?
Are the results consistent, can they be repeated?
In order for it to be a true scientific conclusion, the results must be consistent and repeatable.
As in this example: Saturated fat has been linked to heart disease.
Except in: The Swiss, the French, the Inuits, the Masai, the Spanish and other ‘paradox’ populations.
Or: Red meat is linked to colon cancer...except for another ‘study’ that shows that vegetarians are more likely to get colon cancer.
Neither of these statements are consistent or repeatable.
Who are the real subjects of the study?
In studies with animals and diet, is the diet a natural diet for the animal? How then can you translate any of those results from say, mice to humans? Can the results of the study be transferred to the general population? We have to realize that many test animals that have adverse effects from a particular diet are not being fed the diet they are meant to eat. It may seem obvious that mice do not typically eat dairy products or saturated fats, yet we still hear about the adverse effects these animals may have from this type of diet. Of course, mice would never eat these foods in nature!
What is the ‘difference’?
In science, absolute change is found by subtraction - this is where the results should come from. However, if scientists don’t see a big enough difference this way, they take the results and use division to come up with a more impressive number.
For example, the Lipitor drug trials used two groups of men who were at high risk for heart disease. Lipitor was given to one of these groups for ten years. At the end of the study, 2 men out of the 100 men in the Lipitor group had heart attacks. And, 3.05 out of the 100 men in the control group had heart attacks.
Significant findings? The difference is 1 less heart attack out of 100 men in 10 years. Obviously to most people that’s not an impressive number.
But, if you look for the relative change, (2.0 divided by 3.05=0.64 and 1.0 minus 0.64=0.36). Thirty-six percent—a far more impressive looking number! So we hear Lipitor reduces the risk of a heart attack by 36%.
Have you ever heard the word ‘significant’ in a study? To most of us, significant means impressive, major, meaningful, or important. Significant to a scientist means, ‘not due to chance.’ That’s it.
As in this example: A study was done on two groups of people on the Mediterranean diet with one group adding large amounts of sodium in their diet. The other group ate the same food, but low sodium. What was the difference in blood pressure of the two groups? Scientists called it ‘significant’. BP in the high salt diet group was 126/81. BP in the low sodium group was 123/79. Do YOU Call that significant? I don’t.
Did the researchers control the variables?
Example, “red meat causes cancer”. Really? Well did you know in this study, red meat was grouped in with processed lunchmeats, pizzas, and hotdogs. There is no connection with cancer and red meat by itself.
Compared to what?
Example, “whole grains prevent diabetes”. Compared to what? Compared to white processed flour, yes. Compared to no grains? No, whole grains do not prevent diabetes.
Another example: Filtered cigarette smokers have a lower rate of cancer. Compared to what? To unfiltered cigarettes, true. Compared to non-smokers, false!
Do the results support the conclusion?
Take this example of low carb/high fat diets compared to high carb/low fat diets: The low carb diet group lost more body fat and showed the greatest improvements in all cardio vascular markers including cholesterol, triglycerides, HDL, LDL.
What the scientists published: “Moderate approaches such as a moderate carbohydrate, low fat diet may be prudent.” Could it be because this study was funded by Kellogs?
The media, the medical community, and scientists do not always give us the whole truth. We cannot faithfully believe or follow every so-called ‘study’ we read about or hear about as the truth until we dig a little deeper. As much as we may want to trust medical doctors, the studies and research we hear about from them may be just as flawed as that reported by the media.
We need to apply critical thinking to what we read and see and find out the truth for ourselves. The answer may not always be the most obvious thing, but only after careful evaluation and questioning do we uncover the real truth.
“Science for Smart People”, Tom Naughton, youtube video.
David Freedman, Lies, Damned Lies, and Medical Science, Atlantic Monthly, November 2010.