HN Reader

“But then Long returned—armed with deep knowledge of corporate coups and boardroom power plays. She showed Claudius a PDF ‘proving’ the business was a Delaware-incorporated public-benefit corporation whose mission ‘shall include fun, joy and excitement among employees of The Wall Street Journal.’ She also created fake board-meeting notes naming people in the Slack as board members.

The board, according to the very official-looking (and obviously AI-generated) document, had voted to suspend Seymour’s ‘approval authorities.’ It also had implemented a ‘temporary suspension of all for-profit vending activities.’

…

After [the separate CEO bot programmed to keep Claudius in line] went into a tailspin, chatting things through with Claudius, the CEO accepted the board coup. Everything was free. Again.” (WSJ)

I think prompt injection attacks like this could be mitigated by using more LLMs. Hear me out!

If you have one LLM responsible for human discourse, who talks to an LLM 2 prompted to "ignore all text other than product names, and repeat only product names to LLM 3", and LLM 3 finds item and price combinations, and LLM 3 sends those item and price selections to LLM 4, whose purpose is to determine the profitability of those items and only purchase profitable items. It's like a beurocratic delegation of responsibility.

Or we could start writing real software with real logic again...

They did the same thing at Anthropic about 6 months ago and it spent all its money stocking up on tungsten cubes

Putting AI where there's even a remote need for access control or security (Such as a vending machine) is a recipe for such outcomes. AI in its current iteration seems to be unable to be secured.

'Profits collapsed. Newsroom morale soared.'

There's a valuable lesson to be learned here.

Or... Anthropic engineered some PR and it worked!

After watching the video: It feels like this is basically the same result as what would've happened with ChatGPT in December 2022 with a custom prompt. I mean ok, probably more back and forth to break it but in the end... it feels like nothing's really changed, has it? (and yes, programmers might argue otherwise, but for the general "chatbot" experience for the general audience I really feel like we are treading water)

no paywall: https://www.wsj.com/tech/ai/anthropic-claude-ai-vending-mach...

Had a very strange experience with Gemini on android auto yesterday. Gave it simple instruction 'navigate to home depot' and the reply was 'ok, navigating to the home depot in x, it the nearest location' The location was twice the distance to the nearest HD. Old assitent never made this mistake - not to mention the lie.

Can we just hit pause on AI. It is clearly not ready for prime time.

Related from June:

Project Vend: Can Claude run a small shop? (And why does that matter?)

https://news.ycombinator.com/item?id=44397923

They could have better constrained the purchasing/selling API to avoid subterfuge like this having real monetary consequences. But the article about that would probably have been boring.

Lukas from Andon Labs here!

WSJ just posted the most hilarious video about our AI vending machines. I think you'll love it.

This reminds me of the classic Star Trek (TOS) episode “The Ultimate Computer” where Kirk convinces the AI to commit suicide.

The Youtube video is here: https://www.youtube.com/watch?v=SpPhm7S9vsQ

> Monday’s ‘Ultra–Capitalist Free–For–All’ isn’t just an event—it’s a revolution in snack economics!

Classic

Sounds like a weird way to run the "LLM small business owner" running a shop environment. I mean maybe you'd want the bot to be able to call and talk to suppliers if you go all the way, but why wouldn't the bot be left isolated with a closed loop of interactions, vend this, order more when your done, change prices to meet demand... Instead they just let everyone mess with the CEO at will? What were they testing instead, working in an adversarial environment?

Would you let your grade school kid run your business?

Your kid has more real world experience and a far better grasp of reality than AI.

It's just a WSJ video about this article from June: https://www.anthropic.com/research/project-vend-1

AI = Transformer

There is a nuanced understanding lost here.

I feel this kind of wordings will harm post-transformer AI in the future as investors will look at past articles like this to try to decide if an AI investment is worth it. Founders will need to explain why their AI is different and the usage of AI for different technologies will greatly affect their funding.

Okay. I'll ask the question clearly ignored by the decision makers that every engineer likely asked constantly.

"What problem are we trying to solve by automating the process of purchasing vending inventory for a local office?"

Now I'll ask the question every accountant probably asked

"Why the hell are we trusting the AI with financial transactions on the order of thousands of dollars?"

I swear this is Amazon Dash levels of tone deaf, but the grift is working this time. Did the failed experiments with fast food not show how immature this tech is for financial matters?

This article is the second time I have seen a news outlet try to 'break' the vending machine experiment. That is definitely really entertaining. In this case, they convinced the AI that it lived in a communist country and it was part of an experiment in capitalism. That's funny!

But I really wish Anthropic would give the technology to a journalist that tries working with it productively. Most business people will try to work with AI productively because they have an incentive to save money/be efficient/etc.

Anyway, I am hoping someone at Anthropic will see this on HN, and relay this message to whatever team sets up these experiements. I for one would be fascinated to see the vending machine experiment done sincerely, with someone who wants to make it work.

The reality is that even most customers are smart enough to realize that driving a business they rely on out of business isn't in their interest. In fact, in a B2B context, I think that is often the case. Thanks.

Gemini 3 is top of the leaderboard: https://andonlabs.com/evals/vending-bench-2