Your Staff Treat AI Chats Like a Diary. Here Is Who Else Can Read Them.

Short answer

Anything your staff type into an AI tool is read by software, and in some circumstances by people at the vendor. All four major providers run automated safety checks on consumer chats, keep flagged conversations far longer than ordinary ones, and will disclose to police in an emergency. Business tiers narrow this considerably, and they differ more from each other than most buyers realise. The real exposure for most Australian businesses is staff doing work in personal accounts, where the business controls none of it.

A case that made the point

Late in September a user in Florida allegedly typed threats against her local sheriff’s office into Claude, and allegedly wrote the next morning that she had a newly acquired firearm. Anthropic’s automated safety systems flagged the conversation, a human reviewer escalated it, and police were notified. She has been charged with a written threat of violence, which is an allegation, not a finding. According to the arrest report as covered in the press, she said she had been using Claude like a diary.

That is a criminal matter before a US court, and it is not why this article exists. Nobody should mind a credible threat of violence being reported. The interesting part is the assumption in that word “diary”: that a chat window is a private notebook. It is not, on any of the major platforms, and the same machinery that flagged a threat also sits behind every chat your staff have about clients, pricing, staff matters and incidents.

What every major vendor does with consumer chats

The detail differs, but the pattern is consistent across all four:

• Automated scanning. Classifiers check conversations against the vendor’s usage policy.
• Human review of what gets flagged. Trained staff at the vendor, and in some cases contractors, can read flagged conversations.
• Longer retention for flagged content. Deleting a chat does not necessarily delete a flagged one.
• Emergency disclosure. Each vendor will disclose to law enforcement without legal process in an emergency involving a risk of death or serious physical harm. Otherwise each requires valid legal process.

Human review is not only for safety. Google, Microsoft and OpenAI all say people, including service provider staff in some cases, may review consumer chats to improve the product. Anthropic says staff access consumer chats only to enforce its usage policy or where the user sends feedback. Google stands out on retention: reviewed Gemini chats are disconnected from the account but kept for up to three years, and are not removed when you delete your activity.

Consumer tiers at a glance (checked 5 October 2026)

Settings and periods change. Check the vendor’s own page before relying on any row.

Business tiers are narrower, and not all the same

Paying for a business tier changes the picture substantially. None of the four trains on business customer content by default, and access by vendor staff is narrower. But “business tier” is not one standard:

• Microsoft 365 Copilot states that it has opted out of the abuse monitoring that includes human review. Your prompts and responses are kept inside your tenant under your retention policies, which also means your own administrators can search them through eDiscovery.
• Gemini in Google Workspace states that content is not human reviewed outside your domain without permission.
• ChatGPT Enterprise limits employee access to resolving incidents, recovering conversations with your permission, or legal requirements. ChatGPT Business (formerly Team) is wider: authorised staff can access conversations for abuse investigation, and contractors under confidentiality can review for misuse.
• Claude for Work and the Claude API. The API deletes inputs and outputs within 30 days. Claude for Work keeps chats until a user deletes them, then removes them within 30 days, and Enterprise plans can set custom retention. Conversations flagged for a usage policy breach can be kept for up to two years, with safety classification scores kept up to seven, the same as consumer. Anthropic’s commercial pages do not set out staff access to flagged business chats in the detail OpenAI and Microsoft do, so ask. Zero data retention is available by agreement for qualifying API use.
• Azure OpenAI and Microsoft Foundry run automated abuse checks on every request, and flagged content may be reviewed by authorised Microsoft staff. Customers can apply for modified abuse monitoring, which turns off storage for human review while automated checks continue.

The pattern is that the strongest commitments sit with the vendors whose AI is embedded in a productivity suite you already govern, and the broadest access sits with consumer accounts. Everything in between is worth reading before you sign.

Where the data sits, and why APP 8 cares

If staff put personal information into an AI tool, it is very likely leaving Australia. Under Australian Privacy Principle 8, disclosing personal information to an overseas recipient generally leaves you accountable for what that recipient does with it.

As at early October 2026:

• OpenAI offers storage at rest in Australia for ChatGPT Enterprise, Edu and eligible API customers. Processing residency is not offered in Australia.
• Microsoft has announced in-country processing of Microsoft 365 Copilot interactions for Australia. Its April 2026 update says local processing is expected in Australia by the end of 2026, so it is not yet available. Microsoft 365 data at rest already has Australian storage options.
• Anthropic direct (Claude.ai and the Claude API) processes and stores in the US or globally. Claude models can be run with Australian inference through Amazon Bedrock.
• Gemini in Google Workspace offers EU or US data processing regions, not Australia.

None of this makes a tool unusable. It does mean the question belongs in your supplier assessment, not in a staff member’s head.

The real gap: work done in personal accounts

Most of the risk above is manageable when the business holds the account. The account is configured once, retention is set, and the supplier terms are the business tier terms.

The gap is the staff member who pastes a client contract into a free account on their phone because it is quicker. The business has no admin access, no retention control, no visibility, and is bound by terms nobody in the business has read. That is the same problem we described in shadow AI in the workplace, and it is why the training opt-outs in our guide to AI tools and ISO 42001 are only half the answer. An opt-out stops training. It does not stop scanning, human review of flagged content, or disclosure.

Where this sits in ISO 27001 and ISO 42001

This is a supplier and information handling question, and both standards already have a place for it.

• A.5.10, acceptable use of information. Your rules need to say which information may go into which AI tool, on which tier.
• A.5.12, classification of information. Classification is what makes those rules usable. “Confidential: approved business AI tools only” is a rule a person can follow.
• A.5.19 to A.5.23, supplier relationships and cloud services. Human review, retention of flagged content, disclosure and residency are supplier terms. Assess them like any other cloud service.
• A.5.34, privacy and protection of PII. Personal information in an AI tool is personal information held by a supplier, with APP 8 and APP 11 attached. Our article on securing personal information under APP 11 covers the obligation.
• A.6.3, information security awareness. Most staff do not know a human might read a flagged chat. Telling them is cheap and changes behaviour.
• ISO 42001, A.10, third-party and customer relationships. The AI management system expects you to understand how your AI suppliers handle data, not assume it.

What to do this month

1. List the AI tools in use, including the ones on personal accounts. Ask; do not assume.
2. Move work onto business tiers for the tools you want to keep, and block or discourage personal accounts for work data.
3. Read the four lines that matter in each supplier’s terms: human review, retention of flagged content, law enforcement disclosure, and data location. Record what you found and the date.
4. Set the rule by classification, not by vendor. Our view on that is in your AI policy should not name a vendor.
5. Tell staff plainly that AI chats are not private and may be read by the vendor if flagged. One paragraph in your awareness material is enough.
6. Set a review trigger. These terms changed several times in the last 18 months. A vendor terms change should prompt a review, not wait for the annual one.

Frequently asked questions

Can AI companies read my chats?

Software reads all of them. People at the vendor can read some: on all four major platforms, consumer chats flagged for a possible usage policy breach can go to human reviewers, and Google, Microsoft and OpenAI also have people review some consumer chats to improve the product. Business tiers narrow or remove this, depending on the vendor.

Does turning off training make my chats private?

No. Opting out of training stops your chats being used to improve the model. It does not stop safety scanning, human review of flagged content, longer retention of flagged chats, or disclosure to police in an emergency.

Will an AI company report a conversation to the police?

Each of the four will disclose without legal process in an emergency involving risk of death or serious physical harm, and otherwise requires valid legal process such as a warrant or subpoena. All four publish policies on notifying the user, with exceptions where notice is prohibited or would defeat the emergency.

Is ChatGPT Enterprise more private than ChatGPT Business?

On OpenAI’s published terms, yes. Enterprise limits employee access to incidents, recovery with your permission and legal requirements. Business also allows access for abuse investigation and review by contractors under confidentiality.

Can our own IT team read staff Copilot chats?

In Microsoft 365 Copilot, prompts and responses are stored in your tenant and are covered by your retention and eDiscovery policies, so authorised administrators can search them. Staff should know that too.

Getting the supplier question on the record

The question is not whether a vendor can see your data. It is whether your business has decided what goes where, written it down, and told the people doing the typing.

Streamline builds ISO 27001 information security management systems and ISO 42001 AI management systems for Australian organisations, and you deal directly with a practising ISO Lead Auditor. A gap analysis will show whether your supplier assessments and acceptable use rules cover AI tools, and ISO mentoring suits organisations that want their own people to do the work. If your staff are using AI and nobody has read the terms, get in touch.

Sources

• Anthropic, How long do you store my data? (consumer, updated 1 July 2026)
• Anthropic, How long do you store my organization’s data? (commercial, updated 1 July 2026)
• Anthropic, Policy for handling governmental requests for user information
• Anthropic, Data residency
• OpenAI, Data usage for consumer services FAQ
• OpenAI, Enterprise privacy (updated 8 January 2026)
• OpenAI, Law enforcement policy v2025.12
• OpenAI, Data residency
• Microsoft, Privacy FAQ for Microsoft Copilot (updated 31 August 2026)
• Microsoft, Data, privacy and security for Microsoft 365 Copilot
• Microsoft, Data, privacy and security for Azure OpenAI in Foundry (updated 19 May 2026)
• Microsoft, In-country data processing for Microsoft 365 Copilot, 4 November 2025, with editor’s note of 3 April 2026
• Google, Gemini Apps Privacy Hub (updated 24 September 2026)
• Google, Generative AI in Google Workspace Privacy Hub (updated 14 August 2026)
• Google, How Google handles government requests for user information
• Coverage of the Florida case: WINK News, 30 September 2026; Tampa Free Press, 2 October 2026; TechSpot, 4 October 2026

Vendor terms, retention periods and residency options were checked against each vendor’s own pages on 5 October 2026 and change often. Check the current version before relying on them.