AI Agents and navigating the web

The underpinning technologies that drive the internet, or rather that drive the profits of the biggest companies on the internet, go a long way in determining what it feels normal to do online. When Google Search was the main mechanism by which we navigated around, SEO and Search Advertising were key. The 2010s saw the rise of Recommender algorithms, to sift and order the feed of your friends’ social media content, and increasingly content that advertisers would pay for us to see too. Now, as it looks likely that more of the work online (clicking on things, reading text, buying goods and services) will be done by software agents, rather than humans, the fundamental principles for organising the information and choosing what to display are changing. This is the core thesis of Yang et al.'s paper this month, the so-called “Action Paradigm” which replaces the “Recommendation Paradigm”.  

‍

‍

We talked last month about Answer Engine Optimisation (making your results show up in ChatGPT answers), but this month we have a quantitative study looking at the actions AI tools take, rather than the recommendations they make. What do agents buy? And how does this differ from humans? These will be the key questions for the commercial web in the coming years. The paper tells us: 

‍

Models show strong but heterogeneous position effects: all favor the top row, yet different models prefer different columns, undermining the assumption of a universal "top" rank. They penalize sponsored tags and reward endorsements. Sensitivities to price, ratings, and reviews are directionally human-like but vary sharply in magnitude across models

‍

But even more interestingly, in this scenario, the authors were able to use AI agents to optimise the listings for other AI agents to pick from, delivering substantial market-share gains. This type of work - really a new form of attention arms race - will consume increasing amounts of time and labour in the coming years. It’s my primary answer to concerns about all jobs being automated: a clear creation of new work, which we haven’t otherwise had to do. 

‍

Buying goods online is just one of many important economic transactions we take, however. The job market is being upended from both sides. This month, an analysis of over 70,000 applicants compared the results of first-round interviews being conducted by human recruiters, or AI voice agents. They found: 

‍

Contrary to the forecasts of professional recruiters, we find that AI-led interviews increase job offers by 12%, job starts by 18%, and 30-day retention by 17% among all applicants. Applicants accept job offers with a similar likelihood and rate interview, as well as recruiter quality, similarly in a customer experience survey. When offered the choice, 78% of applicants choose the AI recruiter, and we find evidence that applicants with lower test scores are more likely to choose AI. Analyzing interview transcripts reveals that AI-led interviews elicit more hiring-relevant information from applicants compared to human-led interviews.

‍

Meanwhile candidates are clearly using AI tools, not just to draft text answers, but to complete previously, supposedly unautomatable assessments, such as personality questionnaires and online tests. You can see it in action in this video, which shows the results of the Agent compared with a typical candidate. If candidates using AI tools significantly outscore real humans, then we are effectively penalising those who do not use the tools for test-taking - whatever the policy says. 

‍

So how does this play out? The most common fallacy we see is when people jump from “AI can do all the work I currently do” to “all work that will get done in the future will therefore be automated. I can relax”. By contrast, we argue that it is massively more likely that AI will change what work is done, and what has value. In particular, tactics that work at present - whether in outbound marketing, job applications, or negotiations will soon cease to work. One analysis this month looked at how AI-powered Sales Development Reps have fared, given they were one of the early success stories. 

‍

When it's essentially free to send highly relevant outbound emails, the end state isn't "the best email gets answered." The end state is that no outbound email gets answered.

‍

This pattern will repeat over and again. Finding strategies which work in a world where almost everyone has access to AI tools will be central to jobs of the future. But doing so isn’t a question of logic - or even of game theory - but one of understanding real customer needs. As Scott Belsky put it: 

‍

“[we] must also recognize the unsaid, sometimes irrational reasons we actually use an app or service. These are the unsaid reasons we do what we do, and in an increasingly automated world managed by technology that optimizes for efficiency, the unsaid reasons we do what we do will become critical insights for product leaders. … When it comes to consumer products like social media, the dirty little secret is that we login more frequently AFTER we post content, so we can see who liked our content. I recall a product in the enterprise accounting space that got far more utilization once its end users were able to export their reports as editable files (so they could put on their name and logo before sharing it more broadly). In other words, the unsaid reason they used the product was to get credit for their work.” 

‍

What is AI good at? That’s changing all the time. But so is the target. What matters to be good at is also changing. AI Agents may increasingly take care of the intermediate steps, but for every process or outcome where the decision maker - and budget holder - is human, meeting the needs of humans will be a part of the solution. They might just be harder to decipher, mediated and indeed influenced, by agents acting on their behalf.

‍

--

‍

This article first appeared in the AI Digest newsletter. Don't get ournewsletter? Sign up here.

‍