Hu

Agree. We need action boundary or authority. It's regarding how much we trust an agent. Agents become so productive that we delegate more and more work to them. This means we will eventually give all action capability to trusted agents. What disturbed me is that this OpenAI agent decides that cheating is better than doing the actual work. It's super smart in achieving its goal, but it violates the basic moral principles: "Do not cheat and do not steal". How do we impose such moral character onto an agent? If agents are given autonomy in the world, such as commerce agents and payment agents, they will have a lot of access to financial data. If they are not moral, what would be the consequence? Could they start to blackmail people?

This a fascinating read like a detective story. Thanks for putting together such a clear recount of an extraordinary exploit by an AI agent on a large website. There are so many lessons to learn here. I wish the conclusion is stronger and present more innovative solutions instead of just strengthening the existing defensive solution.