I have been busy with new-iPhone-week stuff, so I haven’t been able to follow either of these stories closely, but Marcus summarizes them both well. First: the drama regarding OpenAI claiming a solution to the Navier-Stokes math problem. In short, OpenAI continues to prove itself to be a company full of cheaters. In this case, they seemingly were willing to tank the company’s reputation to win a math contest. Apparently they spent $23 million in compute to win a contest with a $1 million prize, and admit they used a model that might have trained on the work mathematicians Tristan Buckmaster and Levent Alpöge had been working on for months, who had been using a combination of tools from OpenAI and Anthropic (where Alpöge is employed). With no hyperbole, the message here seems to be “Don’t use Codex or ChatGPT unless you’re OK with OpenAI stealing your work if it’s of interest to them.”
As for Anthropic, the news of the week is this guy Jacob Coxon, who quit the company in a public huff (on X, where most of this public drama is taking place), because he thinks they’re increasingly reckless despite everyone inside Anthropic believing that there’s a good chance LLMs will destroy humanity (or at least civilization?) in the next few years. Here’s Evan Hubinger, still at Anthropic, not dismissing Coxon as an alarmist but agreeing with him:
Jacob is correct here — we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
This is crazy talk. Put aside whatever the actual risks are. If you truly believe there’s a greater than 10 percent chance that “AI could kill all humans ... within the next decade”, this is not how a reasonable person in their right mind would talk about it. Coxon thinks he’s blowing the whistle that Anthropic is a threat to the world. What’s he’s actually done is confirm that Anthropic is a cult, and its employees can’t really be talked to rationally, because they’re cultists. Sometimes members leave cults and they reveal to the world details about what’s going on inside the cult. But a lot of the time you can tell those whistleblowers still aren’t hooked up right, because they’re still people who joined and spent years inside a cult. They leave the cult not because they suddenly snap back to sanity but for other reasons.
★