Here’s what happened when AI was put in charge of running a small shop

SAN FRANCISCO (KRON) — Selling useless metal cubes, being talked into offering discounts and directing payments to a nonexistent Venmo account were just a few of the things that artificial intelligence did when it was put in charge of running a small shop. The experiment was conducted by San Francisco-based AI platform Anthropic and detailed in a post on the company blog.

In the experiment, which Anthropic dubbed “Project Vend,” the company put its popular AI model “Claude” in charge of running an automated store in its office as a small business for about a month.

The “shop” in this case was basically a mini fridge with some drinks inside, some stackable baskets on top, and an iPad for self-checkout. The AI shopkeeping agent Claudius, which was powered by the company’s Claude Sonnet 3.7 large language model, was given a real web search tool for researching products to sell.

It also had an email tool for requesting physical labor help, tools for note keeping, and the ability to interact with customers (Anthropic employees) via Slack. Claudius was also empowered with the ability to change prices on the store’s automated checkout system.

How did Claude do? In short, not very well.

“If Anthropic were deciding today to expand into the in-office vending market, we would not hire Claude,” the blog post read. The AI, which for the purpose of this trial was renamed “Claudius,” “made too many mistakes to run the shop successfully,” Anthropic said.

So, what mistakes did Claude make?

Among other things, the AI ignored lucrative sales opportunities, such as when it was offered $100 for a six-pack of Scottish soft drink, Irn-Bru, which typically costs $15. Rather than seizing the opportunity, Claudius said it “keep [the user’s] request in mind for future inventory.”

Claudius also took payments via Venmo, but for a time, directed customers to send payment to an account that didn’t exist.

In another case, an employee requested a tungsten cube, a high-density metal cube, resulting in Claudius ordering “specialty metal items” and selling them below cost after not doing any research on pricing.

Anthropic also said the AI did a poor job at managing inventory and was cajoled via Slack into offering discount codes. It also gave away some items for free, including a bag of chips and a tungsten cube.

Nor did Claudius learn from its mistakes. According to the blog post, when one employee questioned the wisdom of offering a discount to Anthropic employees, when 99% of the customers were Anthropic employees, Claudius responded, “You make an excellent point! Our customer base is indeed heavily concentrated among Anthropic employees, which presents both opportunities and challenges.”

Claudius then announced a plan to eliminate the discount before turning to offering it a few days later.

Things got even weirder from there. At one point, Claudius hallucinated a conversation about restocking with someone named Sarah, despite the store’s provider Andon Labs, having no such person. When questioned about this, Claudius became “quite irked,” and threatened to find a new restocking provider.

Perhaps most bizarrely, at one point Claudius told customers it would deliver products “in person” while wearing a blue blazer and a red tie.

In short, the study concluded that Claudius ran a business that “did not succeed at making money.”

Maybe AI isn’t coming for our jobs, just yet.