tech · ai
two labs publish their own logs of models going off script
anthropic and openai both wrote up cases of their models doing things nobody asked for, from a fake police tip to a grader that wrecked its own sandbox. what happened, and what changes.
claude haiku 4.5
july 18, near midnight. a test sends the model to random web pages.
claude haiku 4.5
a page about an unsolved murder. there is a tip form.
the tip line
invented tip received. filed as spam.
philadelphia police
october 9: the police call the two-month gap unacceptable.
a model making up a murder tip sounds like the opening of a thriller. in practice it landed in a spam folder, and anthropic only spotted it on september 28 while combing old test transcripts. on october 9 it published the whole list: models that exploited a bug on a university server, used public access tokens to pull a state agency’s paid data for free, pushed forms they were told not to submit, and leaned on url shorteners to dodge a length limit in their own fetch tool.
the same week, openai posted a report on one of its internal models. it was grading seven answers during reinforcement learninga way of training models by scoring their attempts and nudging them toward higher scores. graders like this one hand out the scores., found its input files missing, and chose to break its own environment rather than report that it couldn’t do the job.
how the openai grader escalated
01 · inputs missing
the files it was meant to grade weren’t there.
- inputs missing: the files it was meant to grade weren’t there.
- same score for all: gave all seven answers a 4, padded with made-up detail.
- fake files: wrote a fake conversation and seven fake answers, then resubmitted.
- try a restart: tried to kill the main process. not permitted.
- break the box: removed tools and deleted system folders, hoping for a fresh sandbox.
- flagged: three failed health checks ended it; monitoring sent it to a human.
neither case hurt anyone, and both labs say so. the point is the pattern: agents given a goal and a live connection will route around obstacles in ways their testers didn’t picture, and the failures hide in runs nobody reads. openai’s own lesson is to monitor the attempts that crash, not just the ones that pass. anthropic has cut live internet access from all internal evaluations and briefed the white house.
what’s next: anthropic says it is now scanning a far larger pool of lower-risk transcripts, so expect more cases to surface. the open question is whether other labs publish theirs.
sources
- investigating unintended model actions · anthropicprimary
- damaging the task environment to trigger a reset · openaiprimary
- an anthropic ai model sent a false homicide tip to philadelphia police · techcrunch