Early users were in for several surprises when ChatGPT launched.
The biggest advantage was undoubtedly that we received long, meaningful, and personalized answers to our questions (prompts), rather than a list full of blue links, as is the case with Google.
Another surprise was that we got different results when we tried the same prompt several times in a row. We weren’t used to this: on Google, we got roughly the same list of results for the same keyword—though, of course, they were personalized.
This is due to the technology behind AI models (technically speaking, these are probabilistic rather than deterministic software systems), but it gives users—perhaps even you—the impression that something unpredictable is going on here.
If our goal is to maximize our business’s chances of appearing in AI search engines (and it is), we might easily think we have no control over it. But we do.
In this article, I'll discuss how to measure our visibility—and, by extension, our success—in AI search engines.
AI Search Engines vs. Google Search Results
A classic In SEO, it’s clear what we measure and how: we can quantify—and measure with great precision—how our website performs for various search terms (or keywords).
Let’s take the example of a fictional window manufacturer. If you set up a Google Search Console account for your website, you can see—for free and without any strings attached—exactly when and for which keywords your website appeared, what position it held for that keyword, and how many times users clicked on its subpages.
“custom wooden window fabrication” – 500 impressions, 56 clicks, 2nd position.
In fact, with the help of various SEO tools, you can get a fairly accurate picture of your competitors' data.
In contrast, there is still no established method for measuring AI visibility, and there are several reasons for this. For one thing, the companies that develop LLMs do not share any data on user behavior, so we cannot actually have accurate data.
On the other hand, the basis for the measurement is also completely different, since we’re not starting with keywords: we can’t determine in advance exactly what our target audience will ask or how they’ll ask it. Your customer might type into ChatGPT or other AI search engines that “I need three custom wooden windows”, but it could also be this: “Please recommend a company in the Budapest area that manufactures and installs custom wooden windows. It would be a plus if they could provide references.”.
So we don't know for sure what kinds of prompts people are using to try to solve their problems. Furthermore, it's not certain that the same person will use the same prompt in the future.
In fact, AI search engines often generate far fewer page views, since users frequently get their answers right within the platform. For this reason, you’ll also need to be able to measure your online visibility in some other way, beyond just your website’s traffic.
We’re using AI search engines more and more, so we need to find a solution to the problems mentioned above if we want to remain visible in the online space in the future.
Fortunately, there is light at the end of the tunnel.
AI search engines can also be measured, though it's more difficult
Despite the challenges, there are viable solutions available on the market. These include software tools or custom-built tools that allow us to run prompts in large volumes across various AI search engines.
If we do this at regular intervals (ideally several times a day), we can continuously extract the data and answer important questions.
- What response patterns can we identify in the topics relevant to us?
- Where do search engines get their data? From us, from forums, from corporate blogs, or from somewhere else?
- What are the chances that our brand will appear in the responses?
- What is the likelihood that competitors' brands will appear in the responses?
But is there a good metric for this?
“Share of Voice” (visibility ratio) has become the industry standard in recent times.
The visibility rate indicates the percentage of potential AI responses in which you appear among mentions and citations.
So this is a metric you can use to estimate your visibility in the AI search engine. Of course, a more tangible version of AI visibility is when a client or customer comes to you and says, “The AI said you’re the best.”.
This is the best possible situation for your business, but you won't see it reflected in your website's traffic. You'll need to measure other things in different ways going forward.
For example, track your leads (or sales, in e-commerce) precisely. How many visitors did you get in a month, and from which channels?
Because of AI search engines, you need to set up an in-house, personalized measurement system.
More Accurate Measurement = Greater Competitiveness
We can see that measurement has become more complicated in this new environment. For this reason, we need to be more mindful of it.
We won't be able to rely solely on external analytics platforms (such as Google Analytics), and AI search engines don't provide reports, so we can't count on them (for now).
Therefore, there must be a well-established internal system in place where we clearly track and measure results.
Proper measurement is the first step toward success. We can improve and optimize only what we can see.
What AI search engines show us isn’t a black box. However, we’ll have to develop our own metrics to obtain the data we need to make sound business decisions.
Companies that implement this on time and to a high standard will be more competitive than other players in their market.
