Why not just use each model’s API?

Because the API response often differs from the website's. Country personalisation, connected tools, and product-specific routing all change the user-facing answer. Browser-based scraping captures what a real user actually sees.

How often should I run the monitoring?

Daily for active monitoring; weekly for trend analysis. Models update unpredictably, so high frequency catches drift early. A scheduled batch with rotating prompts and countries gives the cleanest time series.

How do I tell a hallucinated brand from a real one?

Cross-reference against the Google SERP for the same query and country. If a brand appears in an LLM response but has zero presence in Google's organic results, treat it as a likely hallucination and flag it in the analysis.

Does this scale to thousands of prompts?

Yes, because the heavy lifting runs on the proxy network rather than your machine. The scaling cost is per-scrape, not per-instance. Budget accordingly — cost grows linearly with prompt count.

Yes — each model is just another scraper ID. Add the constant, plug it into the fan-out, and the rest of the pipeline (parsing, highlighting, analysis) treats it like any other input.

What stack would I use if I didn’t want Streamlit?

FastAPI + a React or Next.js frontend gives you a productionable surface; Streamlit is the right choice when you want to ship the prototype this weekend. The async backend, the scraping layer, and the analysis prompt port without changes.

LLM Brand Visibility Tracking

Introduction

Ask five different LLMs — ChatGPT, Perplexity, Gemini, Grok, Copilot — “what are the best tools for X?” and you’ll get five materially different answers. Different brands at the top, different tools mentioned, different framing. And millions of real users are taking those answers as gospel for product research without ever clicking through to a website.

That’s the new SEO problem. Traditional rank tracking only watches Google. Brand visibility inside LLM responses is the blind spot — it changes by model, by country, by phrasing, and updates unpredictably as models get retrained. This article walks through the architecture of a tool that fixes the blind spot: send a prompt to five LLMs from a chosen country, extract responses, track mentioned brands, compare to Google’s organic results, and run the whole pipeline at scale. Streamlit on the front, Bright Data’s SERP API on the back, async Python in between.

📚 Table of contents

Why LLM visibility is the new SEO problem
What the tool does, end-to-end
Why API calls aren’t enough — you need a real browser session
Architecture overview
The async fan-out: running five LLMs in parallel
Parsing LLM responses into something comparable
Adding Google SERP for the comparative analysis
Comparative analysis prompt design
Batch mode and the prompt generator
Common mistakes & pro tips
Frequently asked questions

📈 Why LLM visibility is the new SEO problem

Google rank tracking has been a solved problem for two decades. Vendors, dashboards, APIs — the market matured a long time ago. LLM visibility is the opposite: barely measured, deeply consequential, and changing under your feet.

Multiple models, multiple answers. ChatGPT, Perplexity, Gemini, Grok, and Copilot don’t agree on much. A brand at #1 on one is invisible on another.
Country sensitivity. Ask from the US and you get a US bias. Ask from Vietnam and you get Vietnamese apps. The same prompt, different worlds.
Phrasing sensitivity. “Best running shoes” and “top-rated running shoes 2022” can return different shortlists.
Unpredictable drift. Models retrain. Indexes update. Yesterday’s ranking is not today’s ranking, and nobody publishes a changelog.

The practical implication: if you build anything that touches brand visibility — marketing platforms, competitive intel, SaaS for agencies — you have a new tracking problem to solve. Capturing what LLMs say about a brand, at scale, over time, across countries, is the product opportunity.

🛠️ What the tool does

The Streamlit app lets you do four things:

Send a prompt to up to five LLMs in parallel from a chosen country.
Highlight tracked brand keywords inside each response.
Pull Google’s organic results for the same query.
Run a comparative analysis — AI vs Google, gaps and recommendations.

Layer on a batch mode that generates dozens of prompt variations with AI and runs them all concurrently, and you have a tool that turns “how visible is brand X across AI search” from a manual chore into a daily dashboard.

🌐 Why API calls aren’t enough

The intuitive way to query ChatGPT or Perplexity is the API. That fails for this use case:

API responses can differ from the website’s — different routing, different system prompts, different connected tools.
API calls don’t expose the country-level personalisation that the browser experience does.
You can’t see what a real user actually gets unless you mimic a real user’s session.

The fix is a SERP-style scraping API that opens a browser, navigates to ChatGPT (or Perplexity, etc.), types the prompt, and parses the response back. Bright Data’s SERP API does this through a residential proxy network — the request looks like a real user in the chosen country, which is exactly the input you want for visibility tracking. The other lever it gives you is concurrency. Running 100 browsers locally is painful; running 100 through a managed proxy network is a config setting.

🏗️ Architecture overview

The end-to-end flow has five clean stages:

Streamlit UI — prompt, model selection, country selection, tracked-keywords list. Sidebar loads past runs from JSON.
LLM fan-out — run_all_llms() kicks off async tasks against each scraper ID for ChatGPT, Perplexity, Gemini, Grok, and Copilot.
Bright Data SERP API — opens browsers in the chosen country, issues the prompt, returns a snapshot. The app polls the snapshot endpoint until results are ready (typically 20–30 seconds).
Normalise & parse — each model’s output is classified as list-based, ranking-based, or paragraph-based. Tracked keywords are highlighted, and the structured result is stored as JSON.
Comparative analysis — Google SERP results are pulled via the same provider, passed alongside the LLM outputs to OpenAI with a structured analyst prompt, and the markdown result lands back in the UI.

👉 Two API tokens unlock the whole stack: Bright Data (for both LLM scrapers and the Google SERP) and OpenAI (for analysis and prompt generation). Everything else is glue.

⚡ The async fan-out

The whole point of the architecture is concurrency. A single user prompt becomes five concurrent browser sessions; a batch run becomes 50 or 500. Local browser automation maxes out at a few dozen tabs and a long wall-clock time. By offloading to a proxy network, you can issue 100 concurrent sessions and pay only for the scraping — not for spinning up infrastructure to drive them.

Practical patterns:

Wrap each model call in asyncio task; gather them. Don’t serialise.
Treat each LLM’s scraper ID as a config constant. New model? New constant, same code path.
Poll the snapshot endpoint with backoff. The trigger is fast; the wait is for the browser session to finish.
On failure, the run still surfaces partial results for the models that did succeed. Don’t let one timeout kill the whole batch.

🧾 Parsing LLM responses

Each model has its own preferred response shape. Perplexity skews toward ordered lists. ChatGPT favours flowing paragraphs. Gemini sits in between. To compare them, you have to classify the shape first.

The parser does three things:

Detect the response type — list-based, ranking-based, or paragraph-based.
Extract mentioned brands by scanning for tracked keywords (case-insensitive, with simple alias handling).
Surface the mentions in the UI as highlights, so you can scan a long response and see at a glance whether the brand made the cut.

Why classification matters: a ranking-based answer where your brand sits at #3 is a very different signal from a paragraph that mentions your brand in passing. The downstream comparative analysis uses the classification to weight findings correctly.

🔍 Adding Google SERP for the comparative analysis

LLM visibility on its own is interesting; LLM visibility versus Google is actionable. The same Bright Data SERP API can hit Google directly (or Bing or DuckDuckGo — same URL pattern, different engine parameter). You issue the query in the chosen country and language, request a few pages of organic results, and store them alongside the LLM outputs.

The interesting data point is the gap. Brands that dominate Google often barely appear on Perplexity. Brands that ChatGPT confidently recommends may not have a single ranking page on Google. Tracking both columns side by side is the comparison the customer actually wants.

🧠 Comparative analysis prompt design

The analysis call uses a long, structured system prompt. Roughly:

You are an analyst whose main job is to report on the visibility of specific brands and tools across AI search engines and traditional search. Tracked terms are: <keywords>. For each engine, list mentioned brands, position/format of the mention, and any sentiment cues. Then produce a side-by-side comparison: where AI agrees with Google, where they disagree, and where brands are hallucinated. Finish with a gaps-and-recommendations section.

All of the LLM responses, the Google SERP rows, and the tracked-keyword list get injected into the user message. The output is a markdown brief that lands as both an in-app view and a saved file. Long, structured prompts beat short prompts here — the model is doing real analysis, not chitchat.

🪣 Batch mode and the prompt generator

Single-prompt runs are useful for spot checks. Real monitoring needs volume. The app ships a batch mode with two helpful pieces:

Prompt generator. Type a topic (e.g. “best food delivery apps”), pick a count, and OpenAI generates that many natural-sounding variations: different phrasing, different intent, different country targeting if you ask.
Bulk run. Take the generated prompts (or your own list) and fire them all against all five LLMs through the proxy network. The Streamlit table fills as results stream back.

Two practical wins from running at this scale: you can see how the brand list shifts as the prompt drifts (best vs cheapest vs fastest), and you can split by country to spot regional patterns. Run it on a daily schedule and you have a time series.

🚫 Common mistakes & pro tips

Mistakes

Hitting raw model APIs and assuming the response matches the website. It often doesn’t.
Running browsers locally and wondering why concurrency caps at 10.
Skipping country-level testing — the same query from two countries can look unrelated.
Treating LLM hallucinated brands as real. Cross-check against Google to spot them.
Hard-coding one prompt instead of generating a spread of phrasings.

Pro tips

Classify response shape (list / ranking / paragraph) before you compare across models.
Store every run as JSON so you can re-analyse without re-scraping.
Run a daily batch with rotating country targets to build a real visibility time series.
Surface hallucinations explicitly — brands the model invents are a finding, not a bug to hide.
Use the same scrape provider for both LLM and Google SERP. One auth, one rate-limit budget.

🎯 Conclusion

The teams that built rank-tracking tools for Google in 2005 are not the teams building visibility tracking for LLMs in 2026 — yet. The category is wide open, the tooling primitives are mature (Streamlit for the UI, a SERP-grade scraping API for parallelism, OpenAI for analysis), and the customer is already feeling the pain.

The hardest part isn’t the architecture. It’s the unglamorous middle: classifying response shapes, handling country-by-country differences, surfacing hallucinations, and producing a comparison that’s actually useful to a marketing team. Nail those, and the rest is glue.

Explore More on DevShelf

Perplexity — Tool Profile
Full review of Perplexity — one of the AI search engines this tool monitors for brand visibility.
MCP Explained: Build Your Own Server
How to wrap this monitoring pipeline as an MCP server — so any AI coding tool or agent can query brand visibility on demand.

Track Brand Visibility in AI Search