A few years ago, you had a relatively easy time evaluating how well known your brand is through its online presence. All you needed to do was search using your primary keywords, determine what position your website appeared in the search engine results, and review your website traffic. While that process remains relevant today, it does not provide all the necessary information.
More and more people are skipping the blue link list today. Many use ChatGPT to find out which is the best project management tool, others use Gemini to compare different brands, and still others will look over the AI Overview section of a Google search and exit without ever clicking on any links.
When your brand appears in these answers, you get exposure, build confidence with consumers, and sometimes acquire new customers. However, when your brand does not appear in these answers, you could be losing market share, but you wouldn’t know it from reviewing your analytics data.
The challenging aspect of using AI for search results is that they are not like search result rankings. They have no fixed top spot, and they do not provide an official dashboard showing how many times you have been referred to, and the answer provided by the system can vary at any time. However, there are ways to assess your visibility in a predictable, consistent manner. This paper provides an overview of how.
Table of Contents
What “Brand Visibility” Actually Means In AI Answers
Before you measure anything, it’s best if you first establish what it is that you are searching for. Visibility isn’t just one thing when it comes to answers produced by artificial intelligence. There are multiple aspects to visibility.
Mentions
Does the AI provide the name of your brand in response to a related question? This is the most basic indication. For example, if a user submits a query such as “the best CRM for small businesses,” and your product does not appear in the results list, then you are experiencing a visibility gap.
Citations
Several artificial intelligence (AI) tools, such as Google AI Overviews, Gemini with Search, and ChatGPT with Browsing, provide links to the source materials that were used to generate answers. When a tool cites your material, this indicates that the tool found your material credible enough to be used as a basis for generating an answer.
If you are mentioned in a tool, but there is no citation provided, this implies that the tool is aware of you; however, the tool did not utilize your website as a resource for providing an answer.
Position and Importance
If the AI suggests five different brands, do you come at the top of the list, or do you fall to the bottom? Is your description limited to a single sentence, or is it a complete paragraph? The order of the recommendations will carry more weight than if you are listed as the last option in a very lengthy list of options.
Sentiment and framing
How does the AI describe you? “A reliable, affordable option” and “a cheaper alternative with limited features” are both mentions, but they do very different things for your reputation.
Accuracy
Is the AI correct in its presentation of facts about you (e.g., pricing, features, where you are located, etc.)? Some AI systems can make mistakes when presenting facts and can also present obsolete information. Therefore, accuracy is as important as having a presence in the AI system’s database.
Keeping these five concepts in mind will help define all of your future actions. Not only are you asking yourself “Is our presence known?”, but you are also questioning “How frequently, how prominently, how accurately, and how positively is our presence depicted?”
Why The Old Metrics Fall Short
Traditional SEO tools focus on rankings and clicks. Although these metrics remain relevant today, they fail to provide data regarding other key aspects of SEO.
The first reason is that AI responses typically satisfy users without having to go through a link. A user could see your company’s name in an AI Overview, keep that in mind, and then search for you individually several days down the line. When they do so, this will appear as either branded search or direct traffic, but not as an explicit referral from AI.
Secondly, the AI’s response will vary. If you ask the same question to ChatGPT more than once, you will most likely receive a response that varies from the first. If you ask the question in another manner, then the list of brands that are displayed may be totally different. Therefore, one check does not tell you much. In fact, you should perform numerous checks over an extended period.
Third, each platform works differently. ChatGPT uses both the training data it received when it was developed and (optionally) searches the internet. Gemini uses information provided by Google. Google AI Overviews use the current search index and are integrated with traditional search results. A brand could rank highly in one platform but scarcely exist in another.
In other words, we are trying to create a basic measuring system that takes into account the different ways we can measure things, and not just rely on a single device or a single experiment.
How To Measure Your AI Performance
Step One: Build a list of prompts that reflect real customer queries
AI visibility tracking is based on having a well-constructed collection of prompts that represent questions that could be asked by potential customers using an AI-powered system. Poorly developed prompts lead to incorrect tracking results regardless of how precise the tracking is.
Gathering questions that are being asked by people who are actively engaged with the topic you want to write about is a great way to find content ideas. This can come from several different areas, including: sales and customer support teams (which get asked questions daily), the search function within your website, Google Search Console query data, people also ask features on Google Search Results Pages (SERPs), Reddit discussions, Quora answers, and comments on competitive blogs.
Then sort them into groups based on what the customer is trying to do:
- Discovery questions, such as “What are the best accounting tools for freelancers?”
- Comparison questions, such as “Brand A vs Brand B: which is better for a small team?”
- Problem questions, such as “How do I stop my online store from losing abandoned carts?”
- Brand questions, such as “Is [your brand] worth it?” or “What does [your brand] do?”
- Local or specific questions, if they apply to you, such as “Best digital marketing agency in Lucknow.”
You need to use at least 30 prompts to start with. It may seem like a great number; however, this will provide a pattern instead of a mere coincidence. Also, make sure to utilize a variety of prompts. For example, if you have only used your company’s name in every single prompt, then you will only know what the AI responds with when you have explicitly asked it to discuss you. It is more telling to find out if the AI will mention you when the user does not prompt the AI to do so.
Additionally, vary the language you use to describe the same things. Individuals often ask the same questions using different terms. For instance, “Best running shoe for flat feet,” “What type of running shoe do I need to get if I have flat feet?” and “Running shoe for overpronation” are examples of similar questions that can produce significantly different results. By including multiple ways of expressing your most critical concepts, you will have a more accurate view of how users perceive your content.
Step Two: Test Across The Three Platforms
After you have created your list of prompts, you should submit them to each platform. For small teams, this can be done manually, and often they will never have to move past this step.
ChatGPT
Always open a new chat each time you submit a question to ChatGPT, so that the answers to previous questions do not affect the outcome of your current question. Record if the answer includes your business/brand name, where in the answer the name is located, and what words were used to describe your business/brand. Test the model’s responses using both search capabilities (if available) and non-search capabilities, since there may be differences in the model’s responses. When using non-search capabilities, the model is responding based on the knowledge it acquired during its training. When using search capabilities, the model is responding based on information it obtained from the Internet.
Gemini
Run the same prompts and record the same details. Gemini will show you where it gets its information, and because it is run by Google, it will usually be able to give you answers based on what Google has learned about your brand — such as your website, your Google Business Profile, and other parts of the internet.
Google AI Overviews
These will show up in most of your Google searches, but not all of your searches will trigger a Google AI Overview. To determine if there’s a Google AI Overview for your search queries, you can search each of your target questions in a standard browser, preferably in a private browsing window to avoid any effects of personalization on what you’re seeing.
For each search query, note if there was a Google AI Overview generated, if your brand name was included in the content of the Overview, and if your website was cited as one of the sources for the Overview. Please note that the results of the Google AI Overviews may be different based on where you’re conducting the searches from (i.e., geographically) and/or which type of device you’re using to conduct the searches. Therefore, it would be beneficial to conduct the searches from the same geographic locations as your target audience.
For each test, record your results into a basic spreadsheet. You may find it helpful to create a layout that includes the following columns:
- Date
- Platform
- Prompt
- Brand Mention (Yes/No)
- List Position (if applicable)
- Cited Site
- Competitor Appearances
- Sentiment (Positive, Negative, Neutral)
- Factual Errors
The layout can be as simple as you would like. The main benefit is being consistent with recording the information.
Step Three: Turn The Raw Results Into Numbers You Can Track
Most teams find that having a simple spreadsheet with notes is good; however, you will also want some straightforward numbers to look at each month. These are typically the ones most teams use.
Mention Rate
For every prompt you submit to ChatGPT, what percentage of times did your brand name show up? For example, if you tested 50 prompts with ChatGPT and your brand name appeared in 12 of those tests, then your mention rate would be 24%. Make sure to calculate the mention rate for each platform individually.
Share of Voice
Out of all the brand mentions in the prompts you have run through the AI, how much of that is yours compared to your competitors? For example, if the AI mentioned a total of 200 brands in your testing and 30 were yours, then your Share of Voice would be 15%. This can be more telling than just looking at your mention rate since it gives you a sense of where you stand relative to the competition.
Citation Rate
How frequently will a citation link back to your site? This indicates that your content has been utilized as a source, and this is a much stronger indicator than simply having your name included in a citation.
Average Placement
In what range do you typically fall when the AI provides a list-based or ranked response? A general idea of your average would be beneficial.
Sentiment Score
Assign a basic description of “positive”, “neutral”, or “negative” to every instance of the word being used. Then determine the percentages of each type. If the majority of your references are neutral, you might have the chance to improve upon how you are portrayed.
Accuracy rate
Of the responses that reference you, what percent of those are accurate? You should also keep track of your errors individually to help find the root cause of each error.
These numbers do not have to be accurate. All that matters is that you collect data in the same manner as you did before; this will create a true trend. For example, if your mention rate increases from 18 percent to 27 percent in three months, this represents a significant increase in performance despite the fact that the actual percentages may not be exact.
Dealing With The Fact That AI Answers Change
The most frequent error in using an AI to find information is to believe that the first response received is the correct one. Since the AI’s responses are probabilistic, the same question asked multiple times may result in different answers.
If you want to be sure about your results, do them again. You should try to run each of your most critical prompts multiple times (e.g., 3-5) and note down the number of times your brand appears. There will be a significant difference between a brand that shows up 4 out of 5 times as compared to a brand that only shows up 1 out of 5 times, even though they could both appear in your initial trial.
Testing regularly is beneficial. For most companies, testing once per month is sufficient. Testing once per week is appropriate when there is an active marketing campaign occurring, or when you have updated content within the last week. You should not worry about short-term changes that occur daily. Search engine models are updated frequently, as are search engine features, and the Internet is constantly changing. Therefore, you need to look at trends that develop over time (weeks and months), rather than trends that develop quickly (hours).
Using Tools To Save Time
While manual testing may be a great starting point, the process quickly becomes exhausting with hundreds of prompts and three different platforms. Many automation tools are being developed today that run your prompts on a regular basis, provide reports of mentions, citations, and share of voice. Some of these tools are add-ons to established SEO platforms, while others are independent software applications specifically created to help users understand their visibility in the world of Artificial Intelligence.
Because the technology in this category changes so rapidly, I will not make product recommendations in this section. Features and pricing are constantly shifting. In evaluating products in this category, please consider asking a few basic questions:
- What platform does each product support?
- Can each product allow you to create your own custom prompt?
- Does each product provide the entire response to a user’s query so that you can view the context of the response?
- How frequently does each product update its data?
- Will each product also monitor your competition?
A product that provides only a numerical score (e.g., 90%) and does not provide the actual responses to users’ queries is difficult to rely upon and difficult to use as the basis for taking action.
Regardless of which option you select, checking the output manually from time to time is a good idea. Automated systems may produce incorrect results; however, there is nothing like viewing the actual result.
Measuring Google AI Overviews With Your Existing Data
The AI Overviews from Google are a bit easier to link with your regular analytics since these exist within Google Search.
Google Search Console is where you start. The impressions and clicks that come from pages shown in AI features will generally be included with your other search performance data. Therefore, you will not have a distinct row showing only the AI performance.
However, there is still much to gain. Examine which search queries generate the highest number of impressions, particularly long-tail questions. It is these types of search queries that will most frequently generate an AI overview. Next, evaluate how your click-through rate (CTR) has changed relative to those search queries during the period of evaluation.
If you observe that impressions increase, while clicks either remain the same or decrease, an AI Overview may have answered the search query prior to users reaching your website.
Additionally, it is beneficial to determine which of your pages have been referenced by other authors. You may search for the same key questions you asked yourself earlier and identify which of your page addresses (URLs) are listed in the references.
Pages that are commonly cited tend to possess similar characteristics: they provide direct answers to the questions posed at the beginning of the page, they are organized in a logical manner, and they originate from websites that demonstrate genuine knowledge about the subject. Recognizing these patterns can assist you when creating content for future use.
Tracking The Traffic That Does Come Through
In addition to sending visitors to your website, some AI tools allow you to track those visits using Google Analytics. Refer to your Referral Traffic to identify sources like chat.openai.com, chatgpt.com, gemini.google.com, and other similar domain names. You may also want to establish a Custom Segment or Channel Group that combines all of these so that you are able to view AI-driven traffic in a single line on a graph over time.
Don’t expect the number of clicks from AI-generated results to be very high, nor should you base the success of the program solely on these numbers. A lot of users who will view your brand in an AI-generated result will probably not click immediately.
They may search for your name at some point in the future. They may even go straight to your website, or they may tell a colleague about you. Therefore, it is important to track branded search volume and direct traffic as well as AI referrals.
If both branded search volume and AI referral rates increase during the same timeframe, then it is likely there is a correlation between the two. However, you will not be able to establish a causal relationship definitively.
You may also use a straightforward question on either your sign-up form or contact form: “How did you find out about us?” and include multiple-choice options such as “AI Assistant (ChatGPT, Gemini, etc.).” This is a low-tech method; however, it is frequently used to identify influences that analytics can not measure. Your customers will share information that is unavailable through your tracking tools.
Common Mistakes To Avoid
A few mistakes come up again and again when teams start measuring this.
- The first error is that you don’t use enough questions. When you only ask 5-10 questions, you will not be able to get an accurate representation of your users’ opinions, and you may make the incorrect assumption based on those results.
- The second way to use the wrong methodology when testing prompts is to test only prompts that include your company’s name. When every prompt includes your company’s name, you will be happy with the results of your tests, but you won’t know anything about how new customers find you.
- The third way is that you should never ignore how accurate a response is. You can create serious issues for yourself by having someone include your company with the incorrect price, a discontinued item, or a service you do not provide. Ensure that you read the entire response.
- Unlike evaluating a rank position, think of this as market research, which provides you with directions and trends, as opposed to providing you with specific numbers.
- You fail at using the metrics you’ve gathered is by measuring but not acting. The purpose of collecting data on how well you’re doing is to identify areas where you can improve. If you find out that no one cites you, that you provide incorrect information, or that a competitor consistently outperforms you, then these findings will help guide you toward making specific improvements to your content and your overall online presence.
What To Do With What You Find
Measurement is only relevant if it causes us to act on what we’ve learned, so here’s a brief way to consider your next steps.
If you find yourself with a low mention rate, it’s likely that people don’t know who you are. Focus on getting yourself known in areas where AI can gather data about you.
This includes creating good quality content on your website, submitting articles to reputable industry journals, writing reviews for other products or services, and participating in online forums related to your niche. Content that provides direct answers to actual questions typically does better than content designed primarily to promote your brand.
If you’re referenced on a page, however, but not used as a citation, then examine the quality of the content and how it’s organized. Pages that provide direct responses, are well-written, contain unique content, and can be quickly scanned will make it possible for AI tools to utilize them.
When there’s a neutral sentiment towards your product/service, review the comments on review sites/forums. Since AI reflects the overall discussion of a brand, the more positive experiences people have with your company, the better the AI can be expected to reflect those experiences as time progresses. Additionally, the more honest reviews people submit, the better the AI will reflect that as well.
If the information is incorrect, determine how it became incorrect. Many times this will be due to an out-of-date page, inconsistent listings, or a third-party site that has old details. Generally speaking, updating your own pages and correcting all of your profile listings on the internet is typically the most effective solution.
If you are currently performing well, continue to assess your performance. AI systems are continually changing, and what appears to be a good position now may no longer exist in the future.
