The Double Agent Problem

“[Some] companies keep things back for the customers who have the patience to keep asking for them.”

My last piece was about what happens when the customers' patience never runs out.

As I described, (OpenAI's) Codex now chases my complaints for me, and it can keep up the request long after I would have given up. I concluded the article by noting that I'd be interested to see how companies react to the customers' bots heading towards them.

Well, one of the first answers about what companies can do is already in. It doesn't involve blocking the customers' bots, but instead looks something like the visitor pass you get at a company's reception: you sign in, say who you're there to see, and the pass only works on the floors the company has authorised.

The idea is to let the customer's bot in, provided it comes through a door the company builds and sticks to the jobs the company chooses to list.

This sounds helpful for the organisation and pretty limiting for the customer. This door is actually the Personal Agent Protocol, and it was announced by Sierra, a start-up that builds AI agents for customer service teams. Its focus is on protecting the company, not empowering the consumer, and its suggestion is that each company publishes a file on its website telling a customer's agent how to connect and which jobs it may do.

Under the protocol, a customer's bot would have to announce itself and say who it is acting for. The company would decide which services it offered the bot. The latest draft of the specification also lets the company list the tasks it has chosen not to offer through the protocol. OpenAI, which makes Codex, has signed up as a design partner, so the very bot that has been securing refunds and compensation for me may soon announce itself at the door and work within the limits the company sets.

OpenAI is not the only bot provider in town, but the support of a major player would certainly change the landscape for the freedom that customers' bots could operate with.

If OpenAI agrees to this protocol, then the opportunity for individuals' bots to represent them on their terms may close as quickly as it opened.

Sierra calls this “a direct, authorized way to work with businesses”, but the company does the authorising as well as writing the list.

In my last piece I asked whether a human adviser was allowed to ask a customer if they were an AI. The protocol solves that for the company by making my bot confess at the door. Today I can tell Codex not to mention that it's an AI unless someone asks. Tomorrow, announcing itself becomes the price of getting in. The company will know it is dealing with a bot before a word is exchanged, and can decide how much effort or reward that bot deserves.

And what is my bot allowed to ask for once it's inside?

Sierra's examples include checking stock and changing an order, but I have never had to complain my way to a stock check. Sierra also describes a customer's bot working through a warranty claim with the company's own agent. Compensation and lower bills aren't among its named examples, and those are the things customers have to fight for. Both of my recent refunds came out of a fight.

Stripe's Kevin Miller is quoted in Sierra's announcement: “When customers send an agent, they expect the same service they'd get themselves.” I'd be surprised if many companies agreed. The easiest thing for a company to write in its file is one line: no compensation through AI agents. That one line would, on paper, shut out exactly the bots that have been costing companies money.

A company that refuses compensation to bots that declare themselves is teaching customers to keep their bots quiet. And a bot doesn't have to use the company's door. Sierra admits that “some agents may not respect the protocol”. Codex can still go into live chat like any other customer, for as long as OpenAI lets it.

Sierra has seen that coming. Shortly after announcing the protocol, it launched fleming-1, a model that listens to a caller's voice and flags it if it sounds synthetic. Sierra's own line is that “the best case is an agent that says who it is”, and fleming-1 is for the calls where that doesn't happen. On the phone the approach has something to work with. A synthetic voice is generated by software, and Sierra says its model listens “beyond how a voice sounds to a human listener”, picking up patterns in the audio that a person would miss.

For now, live chat and email, where Codex does its chasing for me, remain out of reach in Sierra's response, mostly because spotting AI in text is much harder. In January 2023 OpenAI launched its own tool to spot AI-written text. By its own figures it caught 26% of AI-written text and wrongly flagged 9% of human writing, and it was “very unreliable” on anything under 1,000 characters, which covers most live chat messages. OpenAI withdrew the tool six months later because of its “low rate of accuracy”.

A message that reads like AI proves very little anyway. Anyone can ask ChatGPT to draft a complaint and paste it into live chat themselves. A company that refused compensation whenever a message looked machine-written would end up refusing real customers who had help with the wording, and it would struggle to prove which was which. The phone has the same problem. Sierra's own list of synthetic callers includes people who rely on text-to-speech.

All of this puts OpenAI in something of an awkward position ahead of its IPO. Sierra's chief executive, Bret Taylor, also chairs OpenAI's board. If OpenAI agrees to make Codex announce itself and stick to the company's list, then its users' bots start working for OpenAI's partners as well as for the paying customer. A loss of trust on that point could be worrying for OpenAI just as it prepares to go public.

The published specification says that I alone decide what my bot does, and that it must treat what a company sends “as data, not as instructions”. Last time I asked, Codex told me it was happy to keep a complaint open against a cybersecurity company, given how that company had treated me. Next time, a turned bot would still check the company's list before agreeing to anything of the kind. OpenAI can't sit on the fence on this one. It must decide whose side your bot is on.