The New York Times: From AI Doomsday Theories to Founding Anthropic, Dario Amodei’s Obsession and Safety Play

Author: The New York Times

Translator: Jiahuan, ChainCatcher

On an evening in 2007, Dario Amodei, then a Ph.D. student at Princeton University, received a visiting friend. He had just finished reading Ray Kurzweil’s 2005 book The Singularity Is Near and couldn’t wait to share its ideas with him.

In the book, Kurzweil predicted that artificial intelligence would eventually surpass the human brain, ushering in a moment called “the singularity.” With his fingers twirling a strand of hair, Dario told his friend Colin Heckman that this moment might arrive within their lifetimes.

"He was hooked," said Heckman, who attended the same San Francisco high school as Dario. "He said AI must be built the right way, or else we’re all doomed."

At 24, Dario was still a physics doctoral candidate, absorbing vast amounts of information. He became obsessed with reading relatively niche blogs like Accelerating Future and Overcoming Bias, where authors debated the possibility of AI destroying humanity.

He also dove deep into GiveWell’s online forums. GiveWell is an organization dedicated to maximizing the impact of charitable donations, where intellectuals discussed how to earn the most money for charity.

We interviewed more than two dozen classmates, friends, teachers, and colleagues of Dario. Their accounts show that these books, blogs, and online communities continue to influence him today, offering clues to understanding how the 43-year-old CEO of Anthropic views AI. Anthropic is expected to go public this month, potentially making it one of the largest IPOs globally.

Friends and colleagues say the core of Dario’s worldview consists of two intertwined philosophies. Both firmly believe that mathematical reasoning can change the future of humanity.

The first is “rationalism,” which emphasizes viewing the world in a calm, rational manner. Self-taught AI researcher Eliezer Yudkowsky and others embrace this philosophy, arguing that AI will become powerful enough to potentially destroy society.

The second is “effective altruism,” which developed alongside rationalism. It seeks to maximize social welfare through precise mathematical calculations.

Friends and colleagues say that in Dario, these two schools of thought merged into a firm conviction: AI could be extremely dangerous, and only he and like-minded technologists should be entrusted with developing it. This drove him to race ahead of those indifferent to human safety in creating AI, fueling his desire to control the technology.

In an essay published in September calling for a slowdown in AI development, Dario wrote, “Only if we build this technology the right way can we reap its benefits. As long as we can use the time we gain well, it is worth taking extra care to get things right.”

These ideas became the foundation of Anthropic. In 2021, Dario co-founded the company with his sister, Daniela Amodei, and five others. All of them had connections to the rationalism and effective altruism communities. As the company grew, their views gained increasing attention, while Dario consistently and unabashedly championed them.

Photo: Anthropic Co-founder Daniela Amodei, taken at the 2023 San Francisco office

Dario also engaged in discussions on geopolitical issues. He stated that the US should restrict chip sales to China to weaken its competitiveness in AI. He called for AI regulation to prevent the technology from spiraling out of control. During a dispute with the US Department of Defense, he also argued that AI should not be used for mass surveillance of Americans or deployed in autonomous weapons.

"He genuinely believes we face enormous existential risks," said Nathan Avery. Avery competed with Dario in cross-country running in high school and over the years have had many in-depth conversations, though they’ve grown distant recently. "He feels he needs to position himself so he can take action on this."

Dario is a billionaire, and his net worth is expected to grow further as Anthropic goes public. He declined to comment for this report through company representatives.

His ideas have won him supporters, particularly among some professionals in the AI safety field. These individuals believe that Dario is developing AI responsibly.

But it has also made him enemies in Silicon Valley and Washington, with critics including Meta’s Mark Zuckerberg and Nvidia’s Jensen Huang. They attack him as a “doomer” seeking to consolidate power in the AI field in his hands. In September, the US Department of Defense posted on social media: “Americanism, not effective altruism.”

Dario seems aware that his ideas have angered some people, so he has tried to publicly distance his beliefs from rationalism and effective altruism. In 2023, on The New York Times’ podcast Hard Fork, he said he did not consider himself an effective altruist.

"I agree with some of the ideas," he said, but he opposes “dividing people into different tribes.”

"Dario, Relax!"

Dario and his sister Daniela grew up in the Mission District of San Francisco. Their father, Riccardo Amodei, was a leather craftsman, and their mother, Elena Engel, managed the renovation project of a public library. Divorce records indicate the couple separated in 1991.

By eighth grade at the Lorton Alternative School, Dario already stood out. In 1997, he and a classmate were named the "most likely to succeed," both pictured holding dollar bills in the yearbook photo.

While attending Lowell High School, San Francisco’s top public high school, Dario studied voraciously and even took courses at UC Berkeley. Classmates said he pondered ideas like utilitarianism, which argues that people should create the greatest happiness for the greatest number of lives. He also worried that the Y2K bug (when some computer programs used only two digits for years, entering 2000, programs might mistake “00” for 1900) could cripple global computer systems.

Richard Shapiro, a former physics teacher at Lowell High School, said Dario often felt he hadn’t performed well on exams.

"I’d yell at him, 'Dario, relax!'” Shapiro said.

In 2001, Dario entered Caltech in Pasadena, but frequently returned to SF to visit friends and family. On one such visit, he went to a sushi restaurant with Avery and Heckman, and the three discussed utilitarianism. Dario was a vegetarian, but he didn’t think eating crab was problematic because crabs are invertebrates.

During the meal, Heckman posed a thought experiment: If a person kept killing chickens endlessly, under what circumstances would it be justified to kill that person to stop more chickens from being killed?

Avery recalled Dario chuckled softly, then said, From a utilitarian perspective, once the number of chickens reaches a certain point, their lives might become more important than that of a single human.

Friends say Dario once declared himself an atheist, yet he held firm convictions. In 2003, he drove Avery and others to SF to join protests against the US invasion of Iraq.

The following year, Dario transferred to Stanford University, located in the heart of Silicon Valley. This move also brought him closer to his sister, who was then attending high school nearby.

In 2023, he joked about his sister on a podcast: “We’ve always wanted to save the world together.”

Reflections on Time and Life

After graduating from Stanford in 2006, Dario enrolled in a PhD program in physics at Princeton, eventually focusing on applying physics to biology and computational neuroscience. That year, his father, who suffered from hepatitis C, passed away.

Dario threw himself into research on retinal cells and began taking interest in developing technologies to accelerate scientific progress.

Jade Wang, who dated Dario and co-founded a science discussion group with him, said they often discussed whether scientists dedicating their entire careers to studying a single disease could make a greater impact by instead developing tools or methods for the broader field.

She mentioned that Dario brought up his own father: He died before drugs that could treat most hepatitis C patients were developed.

Photo: Ray Kurzweil, author of The Singularity Is Near

By then, Dario had begun aligning with the works of Kurzweil and rationalist leader Yudkowsky. He was particularly drawn to one of Kurzweil’s ideas: advanced technology could help humans transcend their natural limitations. This ideology is known as “transhumanism.”

In 2007, when Heckman visited Princeton, Dario told him that the singularity would arrive before 2040, and humans might one day merge their minds with machines.

Heckman recalled Dario saying at the time: “As time progresses, we will become a new superintelligent AI, or a hybrid of AI and humans.”

Dario was so captivated by this idea that he encouraged friends and his sister to attend the annual meeting hosted by Yudkowsky’s Singularity Institute. The organization is now called Machine Intelligence Research Institute.

In an email from 2008 seen by The New York Times, Dario told a friend that Yudkowsky was “one of the most profound thinkers I’ve ever met,” and “explicitly trying to engineer a singularity beneficial to humanity.”

In an email to The New York Times, Yudkowsky said Dario “currently does not seek or incorporate my advice, and I don’t recall him ever doing so.”

Photo: Self-taught AI researcher Eliezer Yudkowsky

Around the same period, Dario also began corresponding with Elon Musk, founder of the rocket company SpaceX. Two people familiar with their exchanges said Dario told friends he admired the tech mogul and considered working for him, but ultimately decided not to join the space startup.

In 2008, Dario encountered GiveWell while reading economist Tyler Cowen’s blog. Cowen noted that the organization offered “interesting advice” on charities.

Dario became active on GiveWell’s mailing list and forum, discussing how to conduct charity and scientific work in an effective altruism manner. An email seen by The New York Times shows that in 2010, he wrote that he was “considering entering the financial sector so I could donate more money than I currently do.”

That same year, Dario discussed his decision to donate $10,000 in a GiveWell blog post. He was choosing between two recipients: the Stop TB Partnership, focused on preventing adult tuberculosis, and VillageReach, which helps vaccinate children in African villages.

Dario wrote that the Stop TB Partnership’s “cost-effectiveness” was higher because the organization served adults, while “adults have deeper, more meaningful experiences than infants.” He added, “I believe the death of an adult may be two to three times more severe than that of an infant.”

However, after further comparing how the two organizations executed their projects, Dario ultimately changed his mind, believing VillageReach’s work was more effective.

In 2011, he began conducting postdoctoral research at Stanford University on cancer cells. Friends say that in his personal life, he tried taking anti-aging supplements, adhered to a strict swimming regimen, and became interested in reducing calorie intake. Some studies suggest reducing calorie intake can extend laboratory mice lifespan.

"He often talked about the finiteness of time, and that time is the only non-renewable resource," Avery said. He added that Dario was "nearly ruthless with time."

Two people who knew him said that in 2014, Dario was diagnosed with testicular cancer, further deepening his sense of urgency. He underwent chemotherapy and ultimately made a full recovery.

By then, he had established a close relationship with one of GiveWell’s founders, Holden Karnofsky. GiveWell began collaborating with Good Ventures, a charity founded by Facebook co-founder Dustin Moskovitz and his wife, Cari Tuna, which funds effective altruism-related causes.

Dario began providing life sciences funding advice for GiveWell Labs, a joint initiative between the two organizations.

Photo: GiveWell Co-founder Holden Karnofsky (right), taken in 2012

At the time, Karnofsky was dating Daniela. They moved into a shared apartment in SF, and later residents included Dario, theoretical physicist Jared Kaplan, and college dropout Christopher Olah. Dario had previously tutored Olah in math.

Three people who visited the residence said the roommates discussed Yudkowsky’s essays, exploring rationalism and effective altruism. They would also attend Burning Man, an annual counter-cultural gathering in the Nevada desert, sometimes wearing animal costumes.

Kaplan and Olah later co-founded Anthropic with the Amodei siblings. In 2025, Karnofsky also began working for the company.

From Baidu, Google to OpenAI

In the early 2010s, machine learning advances prompted tech companies to ramp up investment in AI. Machine learning enables computers to learn how to perform tasks by analyzing data.

Baidu, a Chinese internet giant, was one of the companies to dive in. Baidu opened a lab in Sunnyvale, California, and hired AI pioneer Andrew Wu as chief scientist.

Dario joined Baidu in 2014, quickly becoming famous for pulling all-nighters in the office and speaking his mind bluntly.

Photo: AI pioneer Andrew Wu, formerly led Baidu and Google AI teams

Gregory Diamos, a researcher who worked with Dario at Baidu, said: "Dario argued fiercely whenever he got the chance."

Even in the early stages of this wave of AI development, Dario was already concerned about the potential dangers of the technology. Diamos recalled that in 2015, Dario told colleagues that AI would soon reach or exceed human brain capabilities, so companies developing AI should not shy away from the possibility of losing control of the technology.

That same year, Dario joined Google Brain, Google’s AI lab, focusing on AI safety. He and five other researchers, including his roommate Olah, published the paper "Concrete Problems in AI Safety" in 2016, outlining how AI could exhibit anomalous behavior if developed improperly.

In 2016, Dario joined a new AI project: the nonprofit organization OpenAI. He led OpenAI’s safety team and another team researching “scaling laws.” Scaling laws describe how AI model performance may improve as model size, training data, and compute resources increase.

At OpenAI: Ideals, Power, and Friction

Seven people who worked with Dario say that during his time at OpenAI, his convictions about the correct way to develop AI became even stronger. He spoke more frequently about AI’s power and potential harms, and took action to integrate his ideas into the technology’s development process.

For example, three days after Donald Trump was elected US President in November 2016, Dario wrote in OpenAI’s internal messaging app that he worried the US was sliding toward authoritarianism.

Messages seen by The New York Times showed he stated he could “not rest easy researching technology with powerful military applications in an authoritarian country.”

Internal messages indicated that Dario also wrote that investor Peter Thiel was “morally culpable” for donating $1.25 million to Trump’s campaign, and OpenAI should no longer accept his money. (Thiel’s venture capital firm, Founders Fund, later invested in Anthropic.)

Dario also helped OpenAI strike a partnership with funders who aligned with his ideas. Three people familiar with the arrangement said in 2017, he connected OpenAI with the charity run by Karnofsky, while personally serving as a technical advisor to that organization.

Karnofsky’s organization, now known as Coefficient Giving, donated $30 million to OpenAI. Karnofsky also joined OpenAI’s board of directors.

Photo: Holden Karnofsky and Daniela Amodei’s wedding

Late that year, Karnofsky married Daniela. According to their wedding website, guests received plush toys, and those who arranged “the most romantic pairing of different characters” earned rewards.

Dario wore a panda headpiece to the wedding; the dress code was “funny formal wear.” In a photo seen by The New York Times, he appears in this outfit alongside Olah, who wore a giraffe headpiece, and his then-girlfriend Cami Clark. He married Clark in 2022.

In 2018, Daniela joined OpenAI, eventually becoming Vice President of Safety and Policy. Some employees soon began questioning the growing influence of the sibling duo.

Photo: Dario at OpenAI in 2017, and Daniela at OpenAI in 2019

Three people familiar with the discussions said that in December of that year, OpenAI researcher Joshua Achiam told CEO Sam Altman that appointing Dario as the company’s Chief Ethics Officer was unwise. Achiam pointed to the Amodei siblings’ growing power and argued that oversight of AI should be shared among people with differing viewpoints.

When Achiam insisted an external expert should be appointed as Chief Ethics Officer, Dario confronted him face-to-face. Insiders say Dario raised his voice, stating no one was better suited for the role than himself.

In 2018, the Amodei siblings took over GPT, which stands for “Generative Pre-trained Transformer.”

Six people who formerly worked there say they explored a safety technique aimed at teaching AI to complete tasks via human instructions.

This approach later became a hallmark of Dario’s career: bringing AI closer to potential dangers, then stress-testing it with safety measures.

OpenAI President Greg Brockman wanted to participate in the GPT project but faced resistance from the Amodei siblings. Two people familiar with company matters said Dario repeatedly told Altman that he would leave if Brockman wasn’t kept off this work.

An insider said Altman found these ultimatums excessive, so he denied the siblings’ demands.

In December 2020, the Amodei siblings and other colleagues left OpenAI. Four weeks later, they founded Anthropic.

Photo: Amodei siblings sharing the stage with Chief Product Officer Ami Vora at Anthropic’s developer conference this May