Wednesday, February 17, 2016

Unwarranted Extrapolation

Our friends at Faunalytics have released more from their study of former vegetarians. One of their purported findings is that people who stop being vegetarian don't end up eating more chickens and turkeys than the average meat-eater. (Although the study of the general population was done by a student at a different time and through a different sampling method than the study of former vegetarians.) If true, this is good news, given that there are four times more former vegetarians than there are current vegetarians.

The data also seems to indicate that former vegetarians who consume no beef or pork don't eat more chicken than former vegetarians who do consume beef and/or pork. However, the Faunalyitics write-up goes on to make a strange extrapolation (emphasis added):
These findings run counter to a common belief that omnivores who avoid beef and pork will, as a consequence, add in greater quantities of other types of meat (particularly chicken and fish). 
I assume the problem here is obvious, and confusing. Why would you extrapolate from 119 former vegetarians who don't eat any beef or pork, to tens of millions of people who have never been vegetarian but who cut back on red meat?

As I've noted elsewhere, the trend of eating more chickens in place of eating pigs and cows has been documented by professional surveys and peer-reviewed studies, over and over:
Moving from red meat to chicken is a well-documented fact. For example: “‘If you look at dietary recommendations put forth by the U.S. Department of Agriculture [and other health institutions], they are to decrease red meat and substitute lean meat, poultry and fish,’ says Daniel [a nutritional epidemiologist at the University of Texas MD Anderson Cancer Center]. ‘We’ve seen in other data that people are gravitating toward poultry.’”  
Finally, the National Institutes of Health notes “[t]he growing preference in the US for poultry, but not fish, as a replacement for red meat.” 
There are contradictory studies on how much chicken is eaten by people who give up red meat entirely. But for people who reduce the amount of red meat they eat—the majority of people who change their diet for health reasons—all the data are absolutely clear: red-meat reducers eat much, much more chicken. For example, in the largest recent study, those who consumed the lowest amount of red meat ate fifty percent more chicken than those who consumed the most red meat. [Aston, L. M., et al. Meat Intake in Britain in Relation to Other Dietary Components and to Demographic and Risk Factor Variables: Analyses Based on the National Diet and Nutrition Survey of 2000/2001. Journal of Human Nutrition and Dietetics 26(1), October 18, 2012.]
I get it: we don't like to think about the fact that our advocacy can actually hurt animals. But the data is incontrovertible: Red meat consumption has an inverse relation to chicken consumption.


Only during a portion of the time from 2007-2013 did this relationship break. In every other year, before and since, it has held: whenever red meat consumption declines, consumption of chickens increases. Massively. One sample of a few dozen former vegetarians who eat no red meat at all has no bearing on the reality of what real surveys, and the actual consumption data, have shown over and over.


4 comments:

  1. Hi Matt, thanks so much for looking over our report so closely. We’re really happy to have feedback on this research. And we’re very appreciative of all you do in terms of thinking about strategy, so we’re especially glad to have your feedback. Just wanted to follow-up on two points below.
    Kathryn

    “Although the study of the general population was done by a student at a different time and through a different sampling method than the study of former vegetarians.”
    - This is a great point to raise as there are differences between the samples that all readers should be aware of. And indeed we spell these out in the report as part of our effort to be transparent, including that: 1) the comparison of chicken consumption is subject to notable limitations given the different ways the food was listed for each sample; 2) the different frequency scales used for each sample present a weakness; 3) the process of transforming a categorical variable into a continuous one creates potential limitations; and 4) former vegetarians/vegans may have downplayed their meat consumption in the survey due to social desirability bias. We also took what steps we could to make the diet data more comparable, including creating a new chicken variable and finding a way to avoid over-inflating the U.S. population sample’s consumption. To ensure there were no serious red flags in our approach, we also had a volunteer from Statistics Without Borders review the final draft prior to release, as well as someone versed in statistics review an early draft, neither of whom raised concerns about the method so we felt comfortable in proceeding. Also of note is that the samples were collected in the same year (2014), both came from a data collection company, and both were considered generally representative of the U.S. population (the representativeness of both are of course subject to limitations, which we note in the report). A further point of note is the student mentioned (myself) is a PhD candidate also the lead author of the Faunalytics’ study.

    “The data also seems to indicate that former vegetarians who consume no beef or pork don't eat more chicken than former vegetarians who do consume beef and/or pork. However, the Faunalyitics write-up goes on to make a strange extrapolation (emphasis added): These findings run counter to a common belief that omnivores who avoid beef and pork will, as a consequence, add in greater quantities of other types of meat (particularly chicken and fish).  I assume the problem here is obvious, and confusing. Why would you extrapolate from 119 former vegetarians who don't eat any beef or pork, to tens of millions of people who have never been vegetarian but who cut back on red meat?”
    - Also a good question to raise. It appears to be a matter of misinterpretation. Faunalytics wasn’t trying to suggest that our results can be extrapolated to all omnivores, indeed on that same page we say: “The findings cannot be extrapolated to all omnivores because those under study are unique in being lapsed vegetarians/vegans.” What we were trying to point to—and perhaps we could adjust the language to make this crystal clear—was that there is a belief about substitution effects and these findings seem to run counter to this belief in this one specific way. Importantly, we weren’t suggesting that our data definitively answers this question for all omnivores and so all other data should be set aside. That’s why we were careful about including the warning because we did not want readers to erroneously assume we were refuting this notion outright or trying to extrapolate to omnivores. In terms of the differences between “avoid” and “cut back,” our data (as we present it here) only speaks to the former so wisely we’re not trying to make claims about the latter, but it's an interesting topic to consider.

    ReplyDelete
    Replies
    1. Thanks so much for taking the time to expand on this, Kathryn. The difficulty in measuring chicken consumption and social desirability bias are especially important to keep front and center. Being honest and thorough and accurate about chicken consumption is very challenging. I once saw someone claim chicken consumption went up 17% (I think it was 17%), but that was based on an industry claim that meals containing chicken went up by 17%. Not the same thing, obviously1
      However, I would again take issue with this language:
      "there is a belief about substitution effects"
      It isn't a "belief" -- it is a documented fact, as has been shown in many different ways. Since you know and admit "The findings cannot be extrapolated...," I can't help but wonder why you put in the "run counter" claim.
      Regardless, good luck with all your future efforts!

      Delete
  2. Thanks for the back and forth Matt. To ensure there wasn't any further confusion we updated the language in the report. Though it’s worth noting that while we think chickens are a crucially important topic, we don’t think the research in this area is black and white. Primarily because (as you acknowledge) it depends on whether we’re speaking about those who avoid/mostly avoid red meat or those who reduce the amount they eat by a slight or moderate amount. Since we were clear that we are only talking about beef/pork avoiders, most of the existing data on the subject (at least what is presented in Veganomics), does not seem to point to substitution effects, which is in line with our findings (knowing we’re just talking about a segment of omnivores). However, we completely agree that had the topic been those who reduce the amount of red meat they eat by a slight or moderate amount (an important topic indeed!), the research seems to suggest substitution effects. So this distinction is quite relevant to our conversation. Again, I’m really glad we had a chance to discuss this, since it’s an incredibly important topic for our movement to closely consider. We’ve publicly released the veg recidivism dataset and with some further analysis we think our data could answer the question of whether there are substitution effects for those who reduce by a slight or moderate amount, so if you or anyone else is interested in mining the data to uncover this we’d be all ears about the findings!

    ReplyDelete
  3. Thanks for writing back, Kathryn.
    In Veganomics, he presents six relatively small studies where people avoid red meat. Two show substitution, two show inconclusive, and two show overall reduced meat consumption. So for people who avoid all red meat, this is completely inconclusive in terms of if there is a substitution effect for people who completely avoid red meat. Throw in the further qualification that they be former full vegetarians, and you are talking about a very fine slice.
    The study that shows clear substitution effects for people who reduce is a very large and widely recognized study. That is in addition to all the other citations of substitution effects.
    Given this and the vast increase in suffering caused by any switch at all to eating chickens, I would argue that our first task as advocates is to do no harm -- do nothing at all that could even risk encouraging substitution. Unfortunately (IMO), this thought isn't shared by many.

    ReplyDelete