Our behaviours online are changing; we search differently and we visit fewer websites. This two-part series looks at the role of a website today and what you need to consider to make yours work best in the new landscape of the AI-powered internet. This first part is about making sure the machines can find, read and understand your site.
Imagine a typical internet user – let’s call her Betty. She doesn’t bother remembering URLs. She types her question into a search engine or an AI assistant. Most of the time, Betty will get her answer straight back where she asked it without needing to visit another website. But if that un-visited website is your website, what does that mean for you and how can you make sure your messaging reaches Betty?
What has changed?
‘About 60% of searches end without the user progressing to another destination site.’ (Bain & Company) The ‘zero-click’ trend is accelerating across all demographics. We now find answers in a search engine snippet, a reply from a Large Language Model (LLM), or a social app’s search feed. This means people will be visiting your website less and you will need to think more about getting your content to your audiences where they are.
Traffic volume is no longer your key metric. You need to appear in these modern results and to do that you need to have your content found by the robots, understood by them, and appreciated by them.
You need to build accessible foundations
Simply, if you design your website to be easy for people to use, it will be easy for AI to use too. You can do this by ensuring your website is intuitive to navigate and fast for a user to find value. And you need to ensure your website is made considering the broadest range of needs, knowledge, reading habits, and abilities. There are 16.8 million disabled people in the UK. (GOV.UK)
Many people will use a keyboard, a screen reader or voice commands to navigate your site rather than a mouse or a finger. You need to make sure your website navigation works in all of these contexts, you need to ensure content makes sense, and you need to make sure a user can complete all the tasks they need to with the least friction. (Those invasive cookie consent banners, newsletter pop-ups, in-the-way adverts and autoplaying videos will annoy and confuse the machines as much as they do users. We know you need them – but be considerate about how they are included.)
Considering how your website is read in these different ways is a great way to imagine how AI crawlers may experience it. Most machine crawlers view your site in a similar way to a screenreader. They read the code; they infer importance; and they ignore your fancy loading animation. The rules for making your site accessible to humans (WCAG) also help make your site accessible to machines.
What should developers be doing on your website?
A website should be designed and structured for complete clarity – and all labelled up for more clarity. Let’s make it easy for AI:
Ensure it is seen
Start by ensuring crawlers can see your site. Check the robots.txt file (the file that tells crawlers which parts of your site they may access) is not blocking access. You can also use it to signal whether AI crawlers may read and show your content in their results, separately from whether they may use it to train their models.
Define entities
Setting entity signals helps the machines identify distinct parts of your site – from pages to data points. For example, linking your company name to your entries on Wikidata, LinkedIn or Companies House tells the machines that your business is a specific, known organisation.
Mark up your content
Mark up content such as FAQs, articles, and business data with the appropriate schema to let the machines find and understand it.
You can talk to your developers (or us) about checking all of this. Although, these are all decade-old systems for instructing the machines – from long before we had them writing our covering letters, and replacing our spouses. What will be next?
What does the future hold?
Markup is changing for a new era of the web where AI Agents will complete whole tasks for the user from start to finish. Rather than a user searching for flights to Japan then having to go to the website to book the seats themselves; an AI Agent will be able to collect and assess the options before booking the flight, sending the tickets, and telling them the name of the driver who will pick them up at their house.
To support this advanced interaction with your website, Google and Microsoft are launching a new protocol called WebMCP (Web Model Context Protocol). In the future, rather than trying to click a button, an AI agent will call a function — asking your site to carry out a task directly rather than clicking through your interface.
‘The web was designed for humans, with buttons and menus meant for eyes and hands. AI agents, which lack eyes and hands, are interacting with the web, creating a mismatch. Until recently, we made them click buttons and scroll pages like humans. The Web Model Context Protocol (WebMCP) is a new W3C web standard designed to solve this problem.’ A B Vijay Kumar, IBM Fellow and Global CTO for IBM Consulting Delivery.
This technology is still experimental and is likely to evolve before you need to implement it but it is a glimpse into the fast approaching future and is something we should keep in mind.
Look smart, the robots are watching
Getting your content into the AI results is vital for getting heard. Your site needs to be accessible to them (see above) but most importantly, your content needs to be worth accessing. Mediocre content will be passed over. You need to be creating the content that is good enough to appear in the results. Rather than trying to be everything to everyone, focus on what is unique to you and what you can do better than others.
You should aim to create helpful and valuable content. Focus on your niche; that might be you, or your service, or your small corner of an industry. Provide expert insight, proprietary data, and information that helps a user complete a task. Avoid producing shallow fluff that is written to pack in keywords as you will risk getting ignored.
Here are some pointers on how to write your content
Let each point work alone
If you want your content to appear in AI results, you should allow your content to stand alone – to be understood when pulled away from your site and displayed by itself. Reading content prepared like this is good for a browsing person, but it also allows your content to be suitable for a search results snippet or an AI answer result.
Signpost and lead in
Use clear, helpful titles that explain what is coming. This helps people to browse your content and find what is relevant to them. It will also help machines locate the content they need to answer a user’s question themselves.
Be efficient
It is helpful to get to the point. Answer questions in the first lines of copy to save losing a person or machine’s attention. Even think about how your answer may be copied, stripped back and repeated.
Be clear
Ambiguity should be avoided. Be clear about what you are saying and make it effortless to understand. You will confuse fewer people and fewer machines.
Convert text to tables and lists
When presenting data, pricing, features, or comparisons, use native HTML tables or bulleted lists. AI loves extracting structured data from tables.
Illustrate your point
Create high quality visuals that are worth seeing. A stock image can add colour to your page but it is unlikely to add great depth. Can you do more? Can you illustrate your point in a way that is so helpful that it would be pulled for a snippet?
Create FAQs
Imagine what questions your readers will have and try to tackle this in your copy. Think about what your visitors ask, and list out the questions and answers. The machines love to hoover this stuff up.
Include citations
Show where you have sourced information and cite any claims and quotes. You need to build trust with the machines.
If you can create great content, both people and the machines will value it higher.
Your website is your home
This makes it all sound like a website will just be a vessel for content to be taken from. It is true that your website attendance is likely to drop. But while you lose a load of casual passers-by, the more detailed and accurate results provided by these evolving search methods will lead to more high-value visits and more people discovering you in new ways.
After someone has discovered your company they will approach your website with a greater understanding, engagement and intent. On your site you need to help them discover, learn and complete tasks as smoothly and enjoyably as possible. Let’s make those interactions brilliant.
It becomes ever-more important to make your website the inspiring brand flagship. Your website is the place where you have control over your truth – where you choose how to present yourselves and frame your communication.
In conclusion
The AI era doesn’t mark the end of the website but it will make life tough for bad ones. Get your foundations right and you give yourself the best possible chance; a site that is accessible and enjoyable for people is a site the machines can read, understand and pass on. That is how you make sure your messaging reaches Betty, even when she never arrives.
At Eight Arms we have always pushed clients to build websites with logical navigation and clear messaging. That need is only more urgent now — it’s the groundwork everything else stands on. If you’d like a second pair of eyes on your foundations, it’s exactly the kind of thing we can help with.
Being found and understood is only half the story. In part two we’ll look at the other half: once the machines have found you, how do people come to recognise, trust and choose you, and how do you build an audience that’s yours to keep?
Read part 2 of the series where we look at how people come to recognise, trust and choose your business when fewer of them ever visit your site, and how to hold on to an audience of your own.
