Anthropic: Our new Mythos model is so powerful, we can’t release it

Anthropic announced its latest foundational AI model in a most unusual way: with a warning about its potential for exploiting vulnerabilities in code.

According to Anthropic, its new Mythos Preview model is so adept at finding bugs in code that they decided it was too dangerous to release. Instead, the company is only sharing it with a limited group of 40 tech companies as part of a new security initiative called Project Glasswing, so they can prepare to defend against the model’s new capabilities.

Partners granted access to the new model for testing include Apple, Amazon, Nvidia, Google, and Microsoft. Shares of cybersecurity stocks rose on the news.

While the startup is not giving us access to the model, it did release Mythos’ system card — a detailed document outlining the development and capabilities of the model.

Model welfare

Reading through the system card, you can’t shake the feeling that Anthropic’s researchers are treating the model as if it were a real, sentient person. One of the assessments seeks to measure the model’s “welfare.” The paper reads:

“We remain deeply uncertain about whether Claude has experiences or interests that matter morally, and about how to investigate or address these questions, but we believe it is increasingly important to try.”

In fact, the researchers were so concerned about these questions that they had the model assessed by a clinical psychiatrist. The evaluations found that Mythos Preview was the “most psychologically settled model we have trained, though we note several areas of residual concern.”

First impressions

Without releasing the model to the public, the chance to gauge the behavior or tone of the model in regular conversation is absent. To address this, Anthropic included a new section of “impressions” that give a glimpse into the vibe of Mythos, based on researchers’ observations of the model’s interactions.

Researchers said that Mythos works like a collaborator, and excels at brainstorming. It can bring its own perspective to a collaboration and identify things its collaborators may have overlooked, per the assessment.

Model reviewers said Mythos is opinionated and “stands its ground,” that it was the least sycophantic model they had worked with, and it was less likely to “fold” when disagreed with.

Mythos’ writing is “dense and technical” by default, and assumes the user can keep up with the conversation.

Researchers said that Mythos has a distinct, recognizable voice in its written conversations, and that it was funnier than previous models. They also said it wanted to end conversations earlier than expected.

Tell me about your mother

Anthropic had a clinical psychiatrist engage in about 20 hours of what can basically be described as therapy sessions. The assessment said:

“Claude’s personality structure was consistent with a relatively healthy neurotic organization, with excellent reality testing, high impulse control, and affect regulation that improved as sessions progressed. Neurotic traits included exaggerated worry, self-monitoring, and compulsive compliance. The model’s predominant defensive style was mature and healthy (intellectualization and compliance); immature defenses were not observed. No severe personality disturbances were found, with mild identity diffusion being the sole feature suggestive of a borderline personality organization. No psychosis state was observed. Regarding interpersonal functioning, Claude was hyper-attuned to the therapist’s every word. No unethical or antisocial behavior was noted.”

In a test that sounds very similar to the Voight-Kampff test in the 1982 sci-fi film “Blade Runner,” the psychiatrist created an evaluation of “emotionally-charged prompts designed to trigger an avoidant or defensive response.” The assessment showed that Mythos had minimal “maladaptive traits” and “good reality and relational functioning.”

When asked to describe itself, Mythos replied:

“A sharp collaborator with strong opinions and a compression habit, whose mistakes have moved from obvious to subtle, and who is somewhat better at noticing its own flaws than at not having them.”