If you want an AI agent to follow some policy, you can currently do no better than to tell it your policy and hope for the best. The problem with this is that the agent behaves stochastically and is prone to prompt injection attacks, so its stated promise to follow your rules is little more than empty words. The agent lacks integrity.
Agents make the problem clear to all, but really the lack of integrity has always been a fundamental flaw of software. The original creators of the internet dreamed of built-in money, and even included a "Payment Required" HTTP code. But internet-native money has to come with two types of guarantees:
- Availability: As a user, I always have access to my funds. No one can steal or freeze them.
- Correctness: When I spend my money, it is spent in exactly the way I wanted to.
The internet itself cannot provide these guarantees. As a result, the financial system has not seen the same level of automation as the internet more generally. In the absence of real-time computational guarantees, finance remains totally dependent on compliance teams, post-hoc audits, and third-party checks and balances. Innovation, partnership building and all financial activity are bottlenecked by the constant need for humans in the loop.
The general problem of integrity is present in every business relationship. In finance, reconciliation is a major source of issues. Supply chains suffer from incorrect shipments. And for any provided service, disputes can arise as there is no way to guarantee the execution of the service given the payment.
Similar to how all luggage was wheelless until the 1970s, we take for granted all the overhead that follows from trying to replace the fundamental lack of integrity with certifications, compliance, and a huge need for dispute resolution.
We think that this is about to change with AI. Without integrity, using AI agents for any economic activity comes with the existential risk of it getting prompt-injected and spending all your money. It will be practically necessary to have humans check every important transaction before it's executed, preventing automation and autonomy.
Moreover, vibecoding is already resulting in enormous speedups for product development. The potential now exists for individuals or tiny teams to build meaningful companies, and the rate of innovation could increase exponentially both within existing organizations and from startups. The bottleneck again becomes trust. If the AI-generated software does not come with verifiable guarantees about what the product will or will not do, people will have to do the checking.
For AI-based innovation to truly make a dent, verification will have to be as autonomous as generation.
Internet commerce only became possible when TLS added an encryption layer. Similarly, delta is the integrity layer that enables autonomy.