Wednesday, June 24, 2026

Thought Experiments Are Harmful

Passionflower, SDBG


  • Would you pull the trolley lever to kill one person and save five?
  • Would you torture one person to save a trillion from stubbing their toes?
  • Would you donate to prevent an unnecessary death today, or use that money have a one-in-a-trillion chance at creating a quadzillion happy robots over the next billion years?

While people have spent countless hours debating these and similar questions, people are suffering and dying from poverty and preventable diseases

While we one up each other with our expected value, individuals are being tortured - individuals we could help. 

Every hour we spend in mental masturbation is an hour we aren't reducing intense suffering. 

Suffering isn't hypothetical. It isn't an argument or a game or a debate.

Counterpoint:

If we just accept some standard philosophy or dogma, like utilitarianism or Veganism, we also do real harm. 

I did that for decades.

Recently, I saw this error in an article arguing against welfare reforms - specifically, slow-growing chickens. The author noted "The [slow-growing] birds experience fewer hours of excruciating, disabling, and hurtful pain than conventional broilers." But he [obv] goes on to note that more slow-growing birds would be needed to produce the same amount of chicken meat as today's torturously-fast-growing birds. 

This is another example of the fatal flaw of utilitarianism. Suffering does not sum

It doesn't matter if our mental sum of suffering across individuals is greater.

Reducing an individual's extreme suffering is what matters. 

However, it took writing of Losing My Religions - specifically the chapter "Biting the Philosophical Bullet" - for me to realize this. 

Sorry.

2 comments:

  1. While I have some minor differences about “biting the bullet” I absolutely agree with your point here. My reasoning is more motivated by a contention that we grossly overestimate our capacity to predict the future, and so anticipated gains should be moderated by steep negative weighting that bottoms out at about 10 years (and reducing all the time) and this also applies to AI x-risk. This is chiefly applicable to predictions about how much we can create change, we can better extrapolate current trends “if we continue doing this, this trend will continue” (but even this is limited—see Malthus). When people say such-and-such will create such-and-such wonderful outcomes for future people, they are often forgetting about reversion to the mean and pendulum effects.

    ReplyDelete