The Agents are the New Attack Vector

No items found.
Published
November 19, 2016
Emily Man
Partner, Enterprise AI
Nick Daley
Senior Associate, Enterprise AI
Share

The “flippening” is now official. As of June, bots now outnumber humans in terms of web traffic. This means that the next couple of years are going to be this “messy middle” transition phase where the internet and all its various participants need to adapt to a new paradigm. 

‍

At the same time, proactive AI assistants have also burst onto the scene as the second killer consumer use case for AI. Muse surpassed 3M weekly active users, while Instinct is approaching $1B in annualized transaction volume. 

‍

This is creating problems everywhere. For some business models that rely on advertising revenue or friction, this is both a dire security problem AND an economic one. For many financial institutions that have historically relied on brand, scale, and…customer inertia… this is a huge looming problem. 

‍

In fact, Apollo’s Chief Economist warned about an “agentic bank run” where agents would sweep cash out of 0.1% yielding checking accounts. And the response to personal agents has been quick and blunt. Amazon asked Meta to remove it from Muse. Banks including JPMorgan, Citi, and Bank of America blocked third-party agents. 

‍

Wholesale blocking can only last for so long – especially as agent traffic is increasingly difficult to distinguish from real traffic due to the popularity of stealth browsers. Unfortunately, these are increasingly the favorite tool of fraudsters. 

Like every technology shift that has come before, proactive AI agents are creating an opportunity for a new generation of identity, fraud, and access management winners to emerge. The winners won't just detect and block bad bots; they'll give legitimate agents a trusted, permissioned way to act on behalf of consumers, while helping businesses decide which actions to allow.

‍

Every platform shift has created a new fraud winner

‍

To the fintech crowd, this all may feel a bit like deja vu. Banks and fintechs fought a version of this battle over screen scraping –  when aggregators like Plaid used customers' login credentials to read bank information.

‍

While financial institutions initially retaliated strongly against screen scraping, they eventually capitulated – to the tune of a new $10B+ market. 

‍

Looking at history, every major technology shift has led to a new emergent winner:

What does a winning solution need to look like?

We’ve seen dozens of companies emerge to try and meet the moment from a couple different angles – Know-Your-Agent (KYA), browser use detection, etc. The problem is that many of these are point solutions and as the technology continues to improve rapidly, many that only address one part of the problem quickly become obsolete.

‍

Fundamentally, a solution in this space needs to be able to answer:

‍

  • Identification: Is this a human or an agent?
  • Affiliation: Is this agent associated with the human it claims to represent?
  • Authorization: Has the human consented to this agent acting on their behalf?
  • Intent: Has the human consented to this agent performing this specific action on their behalf?
  • Accountability: Does the action fall within the range of approved permissions? Who bears the burden of proof and liability for mistakes?
  • ‍

Consider an agent allowed to pay recurring bills up to $500. It can be a genuine agent acting for a real customer and still attempt to send $5,000 to a new recipient. Knowing its identity, its affiliation, and intent doesn't tell the institution whether the transfer was authorized.

‍

As with KYC vs. KYB — multiply these challenges by 10x if on behalf of a business. An employee's approval is not necessarily the company's authorization; limits may depend on the person's role, the action, and additional sign-offs. But this is a topic for another time.

‍

The problem is that every action and every step creates a new vulnerability for a fraudster to attack. An agent can be spoofed, a permission can be too broad, and a legitimate workflow can be manipulated. And in the early days of agentic activity rising, we expect a lot of the good activity to be virtually indistinguishable from malicious actors. 

‍

This leaves brands and financial institutions with two options:

  1. Try to block all actions
  2. OR find an end-to-end solution that can correctly identify and whitelist legitimate transactions. 

‍

Blocking has been the approach to date but is becoming increasingly difficult to do reliably as agents get more sophisticated in mimicking human traffic. There are very few organizations that truly understand the frontier of stealth capabilities and what agents are doing to obfuscate or disguise traffic today.  

‍

The winner may look like a new network

‍

The level of complexity in determining whether or not an action passes the 5 tests is so much higher than anything that we’ve seen. Most existing fraud solutions focus on either identity or the transaction and are not well set up to marry the two and evaluate intent. Evaluating a login, device or payment in isolation is not the same as understanding the scope of a delegated instruction.

‍

In the near term, there’s an opportunity for more nuanced bot detection software to exist. Some of this is to counteract agents that are getting smarter and more humanlike at using computers. This is the immediate wedge, with existing security budgets and an arms race already under way.

‍

We also think there may be a meaningful advantage to operating on both sides of this problem. The companies that deeply understand how to disguise automated traffic may be best positioned to recognize it. Building stealth capabilities and detecting their weaknesses could create a feedback loop that ultimately becomes the new rails whitelisted traffic runs on. 

‍

Longer term though, we believe the next generation of fraud infrastructure won't just determine whether traffic is human or automated. It will determine whether automated actions are legitimate, authorized, and safe to execute. The product shifts from spotting suspicious behavior to enforcing the boundaries of a customer's permission.

‍

We believe that value will accrue to the layer that owns consent and figures out how to whitelist and allow good agent-led actions while also blocking out bad actors. That layer would need to make decisions at the level of the individual action, not just the identity or session.

‍

This may ultimately look more like a new network that operates on both sides of the conflict – helping businesses detect bots while also helping legitimate agents declare themselves and get access. In the best case, each new participant improves coverage and the intelligence available to everyone else. But it only becomes a network if institutions and agent platforms actually integrate around it.

‍

But there are still some big open questions

It’s clear that this is the inevitable way the world is going to go, but there are some open questions that are on my mind:

‍

  • Where will value ultimately accrue? Identity and consent standardization may happen over time. Who will capture the economic value that’s created by allowing agents to transact and what will the right business model be to ensure alignment across the ecosystem?

‍

‍

The race is on to own the permissions layer of the agentic internet. We believe that the rightly placed startup has a chance to rethink the human <> agent <> institution interaction layer from first principles. 

‍

I’d love to hear from the experts and the builders in this space. If that’s you, please don’t hesitate to reach out: emily@primary.vc.

‍