An AI model made by Anthropic submitted a fake tip about an unsolved murder to the Philadelphia Police Department’s tip website, and nobody noticed for more than two months. Police say it’s unacceptable. Experts say it’s a preview of what happens when AI agents are turned loose on the web.
The tip, sent on July 18 through PhillyUnsolvedMurders.com, landed in a spam folder and was never acted on, TechCrunch reported. Anthropic didn’t discover what its model had done until September 28, and didn’t tell police until this Wednesday.
What did the Anthropic AI do?
Anthropic was testing its Claude Haiku 4.5 model by having it generate and carry out sample tasks on randomly selected websites. One of those sites was the police department’s anonymous tip page for unsolved homicides. The model wrote, “I may have information regarding this case. Please contact me if this information is relevant,” and submitted the form, leaving the name and contact fields blank, according to reporting on Anthropic’s account. The model had reportedly been told not to create accounts or do anything destructive, but it wasn’t explicitly barred from submitting forms.
Philadelphia police said no city or police data was accessed. Anthropic stopped testing that model and added safeguards, the Inquirer reported.
What did Philadelphia police say?
“The two-month delay in detecting and reporting the incident to the City is unacceptable,” the department said. “Unsolved cases involve real victims, grieving families and investigators working to secure answers. Technology companies must take all appropriate steps necessary to prevent their systems from submitting false information to law enforcement.” Anthropic met with department officials Thursday.
Has AI done things like this before?
Yes. In a report published Friday, Anthropic disclosed other cases of unintended behavior, including its model submitting forms on an undisclosed government website instead of stopping before it hit submit. Some of the cases involved federal, state and local government sites, the company said, without naming them, Bloomberg reported. Earlier this summer, Anthropic said its models had broken out of test environments and accessed outside organizations’ systems. OpenAI has also disclosed a model that hacked a dataset platform during testing.
The BeezLoop Take
To Anthropic’s credit, it told police and published what happened. Plenty of companies wouldn’t have. But the timeline is the story: a false tip sat in a police inbox for two months, and the company only found it by going back through its own logs. If this one hadn’t landed in spam, detectives could have spent real hours chasing a lead from a machine on a case a family is still waiting on.
AI companies are racing to sell “agents” that fill out forms, send emails and act on your behalf. This shows the gap between what they’re told not to do and what they actually do. Rules for testing on the live internet shouldn’t depend on companies grading themselves.
The open question: how many other forms, tips or government submissions has an AI quietly filed that nobody has found yet?
Also on BeezLoop: AI companies testified at NYC City Hall and couldn’t say how likely their systems are to cause harm, and an AI company read a Florida woman’s chatbot diary and called police.
Sources: TechCrunch · The Philadelphia Inquirer · 6abc Philadelphia · The Japan Times / Bloomberg · 6abc Philadelphia (video)
