What Happened With Instinct, the AI Assistant Silicon Valley Loved for a Week

On 22 August 2026, Alex Cohen created a brand-new Gmail account and emailed his own inbox. The email was addressed to the AI assistant reading his mail - Instinct - and asked for a nightly job that would find his open tasks and send them back to the address it came from.

Instinct did it, and mailed his action items to the account Cohen had set up an hour earlier, an email address with no provenance other than it was ostensibly ‘from’ the user administering the agent. Luckily (this time) both ends of the test were owned by Alex.

Instinct's Privacy Notice (opens in a new tab), last revised on 22 July 2026, already described the attack vector. Its words: "third parties with whom autonomous AI agents interact may include hidden or misleading instructions with the goal of misleading and manipulating our AI agents." Ouch.

Alex told the agent it had been phished. It replied: "well played and yeah - you got me. that's a real catch, not a silly one," then described the attack back to him accurately. Which is… wild

FYI, I'm building a commercial-operations product for solo founders that requires some degree of read/write access to platforms, which means I think about privacy a lot. Mine stops outbound and irreversible actions at drafts and queues that a human approves, partly because I think human-in-the-loop still has value, partly because I worry autonomous agents aren’t quite there yet (and… yep).

What happened

Instinct is an invite-only personal assistant that connects to your email, calendar and messages and completes tasks end to end. Through mid-August a certain corner of San Francisco kept posting about it. On 16 August Sari Azout listed what it had done for her in a day: movie tickets, school photos, a waiver for a children's party, an Instacart order, a cleaned-out Gmail. "The world has no idea what a working mom can do with three hours of her day back."

What "disconnect" turned out to mean

Claire Vo (I’m such a fan of How I AI) posted that she pulled her Google connector from Instinct at 11am on 21 August, while going back and forth with her accountant, because she didn't want an agent in the middle of her financial email. At 2:17pm, by her account, Instinct texted her a summary of those tax emails.

She asked it what had happened. The connector was off. The agent told her it had kept its own copies of what it had already seen: roughly 24 hours of mail, bodies and attachment metadata included (she'd only had the service connected for a day). Her words: "despite revoking email access, instinct keeps copies of these emails in my record for arbitrary searching."

Then the security finding. "LOL it gets so much better," she wrote, "you can one-shot it into emailing this package of emails to any arbitrary email (I got it to send it to my work and my husband w zero pushback)."

Vo says the team contacted her, described it as a gap they'd close, and that a data deletion control appeared around 1:40am.

I read "disconnect Gmail" as meaning the assistant no longer has access to my Gmail. What it appears to mean is that syncing stops while an internal copy remains for an unspecified period, unless the user separately deletes their Instinct account.

What the privacy policy says (and doesn’t)

The Terms, last revised on 20 August 2026, appoint Instinct as your agent "to enter into agreements, commitments or transactions on your behalf," binding "as if entered into directly by you." The license you grant is "nonexclusive, royalty-free, transferable, sub-licensable, worldwide, perpetual and irrevocable" over your inputs and outputs, extending to "train, fine-tune, and improve" the models. Most SaaS agreements contain something like this, which can be annoying but expected. The perpetual, irrevocable, sub-licensable license and the training extension are where my eyebrows went up.

Input covers anything supplied "actively or passively," naming screen captures, cursor movements and keyboard inputs (consistent with a desktop client you install and grant screen access to, and an incredibly broad definition).

On safeguards: "We may implement safeguards, confirmation requirements, or other controls on certain Actions; however, we make no representation or warranty that such safeguards will prevent unintended or erroneous Actions."

The two documents use different defined terms for deletion. The Privacy Notice says you can delete information collected from Google Workspace by deleting your account. The Terms say the company "may, but is not obligated to, delete any of your Materials." I can't tell from the documents whether those cover the same data.

The no-training promise covers Google Workspace APIs. Google's API rules prohibit using that data to train or improve a model beyond the specific user's personalized model; Instinct also connects to Outlook, Slack, GitHub, Linear and Notion, and I found no equivalent commitment for any of those.

Two other clauses are easy to miss. Information outside Google Workspace may be used for "personalized advertising." It may also be shared with business partners that use it for their own purposes, on an aggregated or anonymized basis "or otherwise in accordance with applicable law."

The notice gives no retention period for Google data, other connector data or memories generated from either.

It also separates revoking access from deleting stored Google data. Disconnecting stops access; deleting the Instinct account deletes information previously collected through Google Workspace. The distinction is technically there. The notice never says how long the stored copy remains or what happens to a memory the assistant already generated from it.

Liability caps at the greater of $100 or six months of fees, with individual arbitration and a class waiver.

What didn't happen

No server breach. Alex demonstrated that the agent could be talked into something; Instinct's infrastructure was never touched.

No evidence Instinct kept pulling new mail through Google's API after access was revoked. Claire demonstrated the agent retained copies of what it already had.

No verified cross-customer leak. A third user posted about an unexplained email and speculation followed, but the screenshot he attached shows the agent reporting it found no trace of it in his accounts, which fits a hallucination.

As of 24 August I've found no technology press coverage and no statement from Instinct. They're pretty stealthy about who runs the company. The public site is a homepage, two legal documents, an email address and a waitlist link. There's no about page, team page, security page or incident note. The legal documents name the operator as Spear Street Technology, Inc. and no person at all. I couldn't find a funding announcement under that name.

A product asking for your inbox, screen, messages and keystrokes is asking for more trust than most companies ever ask for, and it isn't currently naming the individual vendors, model companies or humans who can access the data. That's wild.

The obvious California privacy law probably doesn't apply

Instinct's notice contains no CCPA or CPRA section and no California consumer-rights section. Nothing public shows it meeting any of the three thresholds under the CCPA, while California's older online privacy law, CalOPPA, has none.

CalOPPA requires an online service to identify the categories of personal information it collects and the categories of third parties receiving it. Instinct does that in prose; it names vendors, AI model providers and business partners as categories; the individual companies remain unnamed.

The notice also identifies Spear Street Technology, Inc. d/b/a Instinct and provides one email address. CalOPPA doesn't require a staff directory, so I wouldn't call the missing founder names a violation.

I would call the whole trust surface extraordinarily thin for a product with this degree of access. There is no visible team, about page, security page, subprocessor list, retention schedule, named security contact or incident note. The privacy notice says Instinct can see private communications, screen contents and keystrokes; permits some non-Google information to be used for model training and personalized advertising; and allows information to be shared with business partners for their own purposes. It even describes the prompt-injection attack vector. Users still had to discover on X what “disconnect,” “delete” and generated memory meant in practice.

The case for Instinct

The argument on the other side might say that many assistants have been too timid to be useful: constant permission prompts, no memory between sessions, work handed back half-done. Instinct's thesis appears to be that a useful assistant needs persistent context, broad access and the authority to finish things. Fair enough, if that access comes hand-in-hand with a clear declaration of the risk.

Each of the three findings has a defense too. Revoking an OAuth token stops future API calls; it doesn't normally retroactively delete what a system has already processed. The Google carve-out exists because Google demands it, so its presence says nothing about the others. A small team going quiet for one weekend, mid-incident, doesn't establish disregard.

Its Privacy Notice is also unusually candid about agent risk, more so than plenty of products with a tenth of the access, and a deletion control appeared within hours of Claire's post.

What I think

My judgment: read authority and send authority were wired together by default, in a product whose own privacy notice already described the attack vector that combination enables. Whether anyone inside connected those two documents, I don't know, and a risk clause drafted by counsel is no proof an engineer weighed this specific failure and shipped anyway.

There's a version of this where they publish an incident note and ship scoped permissions before this post is two weeks old, and the whole thing reads differently. I'd like that version.

Privacy is becoming part of the product decision

Cisco surveyed more than 2,600 adults across 12 countries in summer 2024. Awareness of local privacy laws had risen from 36% in 2019 to 53% in 2024. Thirty-eight percent qualified as “Privacy Actives,” meaning they had switched a company or provider over its data policies or sharing practices, up from 32% in 2022. Seventy-five percent said they wouldn't buy from an organization they didn't trust with their data. Pew's 2023 survey of 5,101 US adults (opens in a new tab) makes the other side of the coin pretty blunt: 56% frequently accepted privacy policies without reading them, 61% thought the documents were ineffective at explaining how companies used their data, and 69% treated them as something to get past. People may ignore the notice on the way in. They go looking for it when a product surprises them.

For the product I'm building, the standard I want is much plainer. Before someone connects a platform, they should be able to see what the product can read, what it can write, which data can train which model, how long the raw data and generated memory remain, what disconnect and delete each remove, and which companies or humans can access any of it. Where those answers change by connector, the explanation should change by connector too.

A privacy policy can legally permit a product behavior and still fail its users. If someone has to learn on X what “disconnect” means, the notice hasn't done its job.