A Bonita Springs woman who told deputies she uses an AI chatbot like a diary is facing a felony charge, after the company behind it read what she wrote and called the police. Carli Michelle Heller, 30, is charged with making a written threat of violence against the Lee County Sheriff’s Office.
The chatbot wasn’t ChatGPT. It was Claude, made by Anthropic. But the lesson applies to all of them: what you type into an AI isn’t a locked journal, and the company can read it.
What did the Florida woman write to the AI?
According to the arrest report, as WINK News reported, a user identified as Carli wrote on September 26 that she was going to “shoot up” the Lee County Sheriff’s Office. The next day, the same account said she’d gotten a new gun.
Deputies went to her home in Bonita Springs and detained her without incident. Sheriff Carmine Marceno told WINK that Heller later explained she uses AI “like a diary.” She has a court date in November. Like anyone charged with a crime, she’s presumed innocent.
How did the police find out?
Anthropic’s systems scan for threatening content. According to the arrest report, because of how severe the statements were, they were escalated to a human review team, which reported them to law enforcement.
That’s spelled out in the fine print. Anthropic’s usage policy bans using its tools to promote violence, and its consumer terms let the company report a user’s inputs to police. Its privacy policy allows sharing data when there’s a good-faith belief it’s needed to prevent serious harm, Decrypt reported. OpenAI, Google and Meta have similar rules for their chatbots.
Artificial intelligence is a powerful tool, and like any technology, it can be misused. When someone uses AI to make or facilitate a threat, we have to take that threat seriously.
Lee County Sheriff Carmine Marceno
Marceno added that users are “never truly anonymous” in AI chats.
Are your AI chats private?
Not the way a paper diary is. Here’s what to know:
- Companies can read flagged chats. Automated filters look for threats, child abuse and other dangerous content, and humans may review what gets flagged.
- They can share with police. Most AI companies’ terms allow it in emergencies involving risk of death or serious injury, and in response to legal orders.
- Deleting isn’t always erasing. Flagged conversations may be kept longer for safety review.
- If you’re in crisis, talk to a person. Call or text 988, the Suicide & Crisis Lifeline, any time.
The BeezLoop Take
Threatening to shoot up a sheriff’s office, then saying you just bought a gun, isn’t a private feeling. If a person had heard it, we’d expect them to call the police. A company whose systems flagged it shouldn’t be held to a lower standard, and in this case, the system did what it’s supposed to.
But the case should make everyone think about what they type. AI chatbots are designed to feel like a friend or a therapist. They aren’t either one. Millions of people pour their worst days into these apps, and most of what they write will never be flagged. What gets flagged, and who decides, is mostly set by the companies themselves.
The open question: should the rules for when AI companies read your chats and call police be written by the companies, or by lawmakers?
Also on BeezLoop: Anthropic, OpenAI, Google and Meta testified under oath at NYC City Hall.
Related: Trump put his intelligence chief in charge of a new AI task force.
Sources: WINK News · Decrypt · Daily Caller · Anthropic Usage Policy
