An effective altruist asked if One Step had evaluated our current video against a more "standard" pro-veg video. This was our response:
While we’ll be testing a different pro-chicken video against our current one in February, we’ve not tested our message / video against a “standard” message, because we don’t know how to get data that would be meaningful. As you know, we would need to get impact data from everyone who sees the message, not just those who click through.
More importantly, and even more difficult is to get is the longitudinal data necessary to be able to make clear conclusions. Given that 80% of people who go veg go back to eating animals (and thus become spokespersons against ethical eating), it is important (but basically impossible for us at this time) to do long enough studies to see what everyone's full reaction is over time.
This is, of course, why the default for evaluators is to recommend corporate campaigns and other welfare reforms: those are easily measured. But if we want a truly better world, as opposed to one simply less bad, we need to change people's actual purchases, in a realistic and meaningful way.
Our intention, of course, is that by avoiding a veg message, we avoid creating four failed vegetarians for every lasting new vegetarian.