Bill Chameides: Do Organic Foods Promote Better Health?

I agree completely with Bill Chameides' blog post below. There's so much "junk science/journalism" on the web and in the media.

Do Organic Foods Promote Better Health?

A fresh look at a new study on the benefits (or lack thereof) of organic food over conventional.
In the week since Crystal Smith-Spangler of Stanford University and colleagues published their meta-analysis* comparing conventional and organic foods in the Annals of Internal Medicine, I've seen a lot of headlines making some pretty broad claims: "Little evidence of health benefits from organic foods" and "organic food is not healthier" are just two examples.

The study's authors have also been making the rounds, being quoted in the news, as in the case of author Dena Bravata on ScienceDaily.com, like so: "There isn't much difference between organic and conventional foods, if you're an adult and making a decision based solely on your health."

That's a pretty definitive statement. My problem is that I just don't think the paper the authors wrote justifies these very same authors' stating such a conclusion. Let's take a look at why.

The Study Didn't Really Address Human Health Question

Organic foods are more expensive than conventional foods, and so the question shoppers most likely ask themselves during decision-making time at the market is: Do the benefits of organic foods justify the extra cost? For most people, I suspect, that question gets translated into: "Do organic foods lead to better health as compared to conventional foods?" Now, granted, that's not the only question people could ask -- for instance, folks could consider whether organic food production leads to better environmental outcomes. But for the purposes of this post, let's stick with the question that motivated the authors' study -- the question of health benefits.

The problem is that the data used by Smith-Spangler et al. were not gathered to answer that question directly. (Remember, it's a meta-analysis.) For example, the data did not even come from one or more long-term studies that compared the health of one group of people who ate exclusively organic to that of another group who ate exclusively conventional food. Why not? Because those data don't exist.

Instead, the authors combed through thousands of published studies containing data on any aspect of the organic-versus-conventional food conundrum, culled them down to 237 relevant ones, and then compared and contrasted the data from those studies to elucidate areas of agreement, disagreement, overarching patterns and other interesting relationships.* And here's the thing, only 17 of the 237 were with people, and of those 17, only three had clinical health outcomes. The vast majority of the studies -- a whopping 223 -- were "studies of nutrient and contaminant levels in foods." In other words the study had next to no data on the benefits or lack thereof when it comes to human health. And so, despite claims otherwise, if "you're an adult and making a decision based solely on your health," the results of the Smith-Spangler study are only tangentially relevant. That tangential relevance comes from the results the authors obtained with regard to organic vs. conventional food -- which carry a bit more weight. Let's take a look.

No Difference Between Organic and Conventional Foods?

The fact that the authors are telling media outlets that there "isn't much difference" between these two types of food is a bit perplexing, because that's simply not the conclusion of their paper. Here's a direct quotation from their study:

"The published literature lacks strong evidence that organic foods are significantly more nutritious than conventional foods. Consumption of organic foods may reduce exposure to pesticide residues and antibiotic-resistant bacteria."

So while the authors found no evidence of a nutritional benefit from organic foods (e.g., with respect to vitamin content), they did find a benefit related to food contamination -- namely, lower exposure to pesticides and antibiotic-resistant bacteria in organic foods.

Now, the authors note that the levels of pesticides in all foods are generally within government regulations but that conclusion is based on a very small sample. Only three European studies (out of the 237) included data on "the prevalence of contamination exceeding maximum allowed limits." And in any event, when it comes to food regulations, it's pretty clear there are a lot of folks out there who are not convinced they are protective enough.

The Transitive Property of Health?

So, are organic foods better for your health? It probably depends upon what you think about ingesting even small amounts of pesticides and antibiotic-resistant bacteria. Some consumers prefer to limit their exposure to pesticides and antibiotics as much as possible; others not so much. It's certainly something to consider given what we're learning about the environment and health -- how even tiny amounts of certain chemicals (such as pesticides) can affect our epigenome (the part of our genetic code that tells whether a gene should be turned on or off), how antibiotics and other compounds change our microbiome (the collective microbial genomes that inhabit our bodies), and how such epigenomic and microbiome changes can lead to chronic disease. (Read more on this subject in my posts on the epigenome and www.nicholas.duke.edu/thegreengrok/microbiome).

The interesting (and disturbing) fact is that if you count yourself among those who want no part of pesticides in food, the Stanford study likely confirms your belief that organics are better. But if you just read the headlines and media hype on the paper, you would miss that entirely. When it comes to the chemical marketplace, I often advise caveat emptor. The same can apply to the news you digest, as well.

Meta-Analyses -- A Note on Their Strengths and Weaknesses Vis-à-vis the Stanford Study

The Smith-Spangler meta-analysis used statistics to compare and contrast data from different studies to elucidate areas of agreement, disagreement, overarching patterns, or other interesting relationships. Commonly used in medicine, meta-analyses can be a powerful tool. If a finding can make it through the compiling of different studies with different methodologies, then it's safe to say that finding has a certain rigor.

But meta-analyses also have their limitations. Below I discuss three: the heterogeneity of the studies analyzed, mixed quality of the studies' data and common denominators.

Heterogeneity: A meta-analysis is at the mercy of the heterogeneity of the included studies: the more similar the studies, the more valid the analysis; the more heterogeneous the studies, the lack of overlap means you're not necessarily getting multiple looks at the same topic.
Stanford's study included only 17 studies of people, none of which featured a group that ate entirely organic. Five of the 17 studies were conducted on people who ate predominately organic, and the remaining 12 studies were on people who ate only certain organic foods. How do you compare the benefits of organic food on health outcomes when you can't even isolate the groups?

In addition, the studies are highly variable. For example, one Dutch study collected a one-time breast milk sample from more than 300 women to evaluate fat content differences whereas an Indian study (not available online) compared levels of several nutrients (including vitamin C, vitamin K, and calcium) in a sample of six organic versus six conventional oranges. The disparate nature of the studies' data indicates how hard it would be to extrapolate information that could easily answer the study's main question: whether organics lead to better health.

Data Quality:  The strength of a meta-analysis is dependent on the quality of the original studies. Say some of the original studies are brilliant works while others are not so much, those big differences become lost once the studies are grouped together.

In fact, the Stanford authors state that in general the 17 studies of people were of "fair quality."

Common Denominators: To compare data across studies, the authors must define parameters, which must be analyzed and weighted statistically, a process that can introduce ambiguity to the data as well as bias (a criticism of the Stanford study leveled by Chuck Benbrook of Washington State University).

For example, in the Smith-Spangler study, the criterion for defining pesticide exposure was simply either the presence or absence of chemical residues without distinguishing between more and less harmless types of pesticides, concentrations of pesticide, or whether exposures included multiple pesticides.