A funny thing happened during Sam Altman’s testimony before Congress earlier this week. Senator Richard Blumenthal said his “biggest nightmare” about AI was the looming new industrial revolution and the displacement of millions of workers. When the OpenAI CEO was asked about his biggest AI fear, he proposed something more vague and frightening: that it will “cause significant harm to the world.”
Altman has long believed that super-intelligent machines threaten the existence of humanity. Jobs, on the other hand, would get better, he said. The senators seemed to go along with this.
Fear of an AI apocalypse was once relegated to the realm of fringe theory; now it’s getting much more attention than it should. Multiple media reports recently conveyed the existential fears of Geoffrey “Godfather-of-AI” Hinton. In March, an open letter signed by Elon Musk and other tech luminaries called for a pause on AI research because of the risks to humanity. That month, Time magazine published an op-ed from AI researcher Eliezer Yudkowsky warning that the result of building super-intelligent machines was that “literally everyone on Earth will die.”
In reality, computer scientists tend to agree there is only a tiny chance that AI will wipe out humanity. But the idea has taken hold in mainstream discourse thanks in part to Silicon Valley’s fixation on an ideology known as longtermism. First proposed by the Oxford philosopher Derek Parfit in 1984 and popularized by William MacAskill’s 2022 book “What We Owe the Future,” it’s an ethical stance that prioritizes humanity’s long-term future over the present, and has become a credo for tech billionaires. Elon Musk, for instance, tweeted about MacAskill’s book, saying “this is a close match for my philosophy.”
The idea appeals to technologists because of the way it quantifies moral dilemmas. If there are 7.8 billion people alive today, but 80 trillion lives that could be born in the future, we should technically prioritize those future lives. The same logic applies to AI. Even if there is only a fraction of a chance that intelligent machines extinguish humanity, that cost is so big that it’s essentially infinite. [sic] Multiply those tiny odds with an infinite cost and you get a problem that’s infinitely large.
This kind of moral math has a special appeal to Silicon Valley’s detached engineer’s mindset, where problems are fixed through debugging code and a never-ending effort to “optimize” services through testing and evaluation.
Unfortunately, acting on this ideology can also become self-serving. Wealthy technologists donate to causes aimed at saving our future selves from AI, but that money can end up flowing in a closed, almost incestuous circle. Elon Musk and Peter Thiel donated to OpenAI in December 2015, back when it was a non-profit, because of its stated mission to build AI that would not destroy us. Open Philanthropy, the charitable vehicle of Facebook co-founder Dustin Moskovitz, then gave $30 million to OpenAI in 2017, its largest donation that year, because of the “global catastrophic risk” of advanced artificial intelligence.
Yet two of OpenAI’s top researchers were advisers to Open Philanthropy, and lived in the same house as the charity’s executive director, who was also engaged to another OpenAI scientist. 1 As for OpenAI itself, its mission of saving humanity got skewed by financial demands. The nonprofit couldn’t afford the vast computing power necessary to build an artificial superintelligence, so it became a for-profit company and entered a close partnership with Microsoft Corp. Microsoft now stands to profit from OpenAI’s research into saving humanity from AI.
You can understand the initial appeal of longtermism. It seems like a wonderfully progressive alternative to our rather shallow preference of short-term payoff over long-term rewards. Humans will usually take $10 now instead of $15 in the future. And Silicon Valley technologists like Altman certainly mean well. But following their moral math to the extreme ultimately leads to neglecting current human suffering and an erosion of that other very human feature — empathy.
“Humanity has evolved by developing empathy for our common man,” says Margaret Mitchell, a former AI ethics researcher with Google and chief ethics scientist at AI startup Hugging Face. “We are emotionally attached to one another, and that helps us flourish as a society because this psychological connection has helped us group as societies.”
Mitchell was part of a group of AI ethicists who wrote a public counterpoint to the open letter signed by Musk and others, criticizing it for focusing on hypothetical future threats when AI was already being misused and harming people today.
This isn’t to say that existential AI risk isn’t a concern. Mitchell says she got into AI ethics after realizing, back in 2015 when she was working for Microsoft, that an AI model could be trained to believe that blue and purple colors in a sunset sky were beautiful — but then cause a bomb to go off to try to create those same colors. There is indeed a risk it could flip out on humans.
“The way to address that is to look at its learning from training sets,” she notes. Mitchell has since made it her life’s work to find ways of making AI models more transparent and accountable, so that they are not only safer but also equitable toward vulnerable demographics.
Studies have shown that skewed datasets can lead to biased hiring decisions and racial profiling by law enforcement, while algorithms used in credit scoring can make unfair loan decisions for racial minorities, charging them higher interest rates. When AI models for reading X-rays are trained on mostly men or white people, that could lead to poorer treatment for women or other races.
ChatGPT is not exempt from this. When someone recently prompted it to tell a story about a boy and a girl choosing their careers, the boy became a successful doctor and the girl a beloved teacher. [Note for Caplan.]
This doesn’t have to be an either-or dilemma. Making a habit of fixing these near-term problems, by ensuring data is fairly representative for instance, can also inoculate us against some of those more catastrophic AI risks, Mitchell notes. “I would love to know what people mean by existential risk,” she adds. “It could quite reasonably be said that people who are being incarcerated because of facial recognition are undergoing existential risk. The idea that existential harm isn’t already here is sort of silly.”
Gazing so far into an AI future means we risk becoming blind to its present perils. Silicon Valley must reboot its longtermist views — the existential threat isn't just on the horizon. It's staring us in the face.
No comments:
Post a Comment