Measuring AI Citation Impact: Analytics, Logs, and Panel Scrapes

Introduction

AI-driven discovery is now split across several measurement layers. A person may see a company cited in an AI answer, click a link with no useful referrer, visit through a browser that strips campaign data, or read an answer without clicking at all. Meanwhile, an AI crawler may request the company’s page without representing a human visitor.

A useful measurement system must therefore separate four questions:

Was the site crawled or retrieved?
Was the site cited in an AI answer?
Did a person visit the site from that answer?
Did that visit produce meaningful engagement or revenue?

As of September 22, 2026, first-party reporting is improving. Google Search Console provides a Generative AI performance report for AI Overviews and AI Mode, including impressions, pages, countries, devices, and dates. The report became available worldwide on August 31, 2026. Google’s Generative AI performance report (support.google.com)

Microsoft also provides an AI Performance report in Bing Webmaster Tools. Its public preview reports total citations, cited pages, grounding queries, and citation trends across supported Microsoft AI experiences. Bing Webmaster Tools AI Performance (blogs.bing.com)

These tools should be treated as authoritative for their own surfaces, but they do not create a complete cross-assistant measurement system. The recommended approach is to combine first-party reports, server logs, controlled tracking links, analytics exports, and scheduled answer-panel observations.

1. Why AI Referral Measurement Is Difficult

Traditional web analytics depends heavily on the Referer header and campaign parameters. Both can be missing or incomplete.

The HTTP Referer header may contain the previous page, only the previous website origin, or no value at all. Browser referrer policies can intentionally limit or remove that information. MDN Referrer-Policy documentation (developer.mozilla.org)

AI systems create additional complications:

An assistant may link to a page without sending a recognizable referrer.
An assistant may use a redirecting or rewritten URL.
An assistant may fetch content server-side before showing an answer.
An AI crawler may visit a page without any human seeing it.
The answer may cite a canonical URL instead of the URL that was originally retrieved.
The answer may change depending on location, device, language, time, or personalization.

For example, OpenAI distinguishes between OAI-SearchBot for search discovery, GPTBot for possible training use, and ChatGPT-User for user-initiated actions. Perplexity separately identifies PerplexityBot and Perplexity-User. Anthropic identifies ClaudeBot, Claude-SearchBot, and Claude-User. These requests should not be combined into one generic “AI traffic” number. OpenAI crawler documentation (openai.com) Perplexity crawler documentation (docs.perplexity.ai) Anthropic crawler documentation (support.anthropic.com)

The central rule is:

Crawler activity is not referral traffic, citation visibility is not traffic, and traffic is not business impact.

2. The Measurement Framework

Use five measurement layers.

Layer	Main question	Recommended data
Access	Did an AI system request the page?	Web server logs, firewall logs
Visibility	Was the page shown or cited?	Google Search Console, Bing Webmaster Tools, answer-panel observations
Acquisition	Did a person arrive from an AI surface?	Referrer, campaign parameters, short-link clicks, analytics
Engagement	Did the visitor interact meaningfully?	Analytics events, page views, engagement time
Value	Did the visit produce a desired result?	Signups, leads, purchases, subscriptions

Recommended source-of-truth hierarchy

Use the strongest available signal for each question:

First-party platform reports for impressions and citations.
Answer-panel observations for prompt-level visibility and competitive comparisons.
Redirect logs for controlled link clicks.
Web analytics for sessions, engagement, and conversions.
Server logs for crawler and fetcher activity.
Modeled estimates only when direct evidence is unavailable.

Do not combine all signals into a single number without preserving the source and confidence level.

3. Core Data Model

Create a warehouse with separate tables for raw observations and derived metrics.

`panel_runs`

One row for each prompt and surface check.

run_id
observed_at
surface
topic
prompt_id
query_text
locale
country
device
panel_present
panel_type
answer_hash
raw_html_path
screenshot_path
collector_version
status

`panel_citations`

One row for each link observed in an answer panel.

run_id
surface
topic
prompt_id
cited_url
cited_domain
citation_position
link_scope
target_domain
target_cited

`ai_crawler_requests`

Aggregated server-log data.

date
bot_name
bot_role
path
requests
successful_requests
error_requests
unique_ip_count

`shortener_clicks`

One row for each controlled-link request.

click_id
occurred_at
slug
surface
topic
prompt_id
asset_id
referrer_host
user_agent_family
ip_hmac
destination_path

`ai_sessions`

A normalized analytics table.

session_id
date
surface
topic
prompt_id
landing_page
attribution_method
confidence
engaged
engagement_time_seconds
key_event
revenue

The confidence field is important. Recommended values are:

A — Direct: controlled short-link click or trusted campaign parameter.
B — Strong: known assistant referrer or verified user-fetch request.
C — Probable: campaign or landing-page evidence without a complete referrer.
D — Modeled: inferred from timing, panel observations, or aggregate changes.

Report A and B separately from C and D.

4. Metric Definitions

Citation Share by Topic

Citation Share by Topic measures how often a target domain appears among the sources observed for a topic.

[ \text{Citation Share}_{topic}

\frac{\text{Target-domain citation occurrences}} {\text{All citation occurrences}} ]

Example:

100 total citations observed for “project management software”
12 citations point to your domain
Citation Share by Topic = 12%

This metric measures source share, not whether an answer contained your site at least once.

Citation Coverage

Use a second metric for answer-level presence:

[ \text{Citation Coverage}_{topic}

\frac{\text{Answer runs containing a target citation}} {\text{Answer runs where a panel was present}} ]

Citation Coverage is often easier for nontechnical audiences to understand:

“Our site appeared in 34% of observed AI answers for this topic.”

Citation Position

Record the position of a citation within the answer:

1 = first observed source
2 = second observed source
3 = third observed source

Because AI interfaces often group or reorder sources, treat position as a descriptive observation, not a universal ranking.

Assistant Referral Rate

Assistant Referral Rate measures the proportion of human site sessions that can be attributed to an AI surface.

[ \text{Assistant Referral Rate}

\frac{\text{AI-attributed human sessions}} {\text{All human sessions}} ]

Report at least two versions:

Confirmed Assistant Referral Rate
= A-confidence AI sessions / all human sessions

Expanded Assistant Referral Rate
= A + B + C-confidence AI sessions / all human sessions

The confirmed version is more conservative. The expanded version is useful for trend analysis but should not be presented as fully observed traffic.

Do not use panel runs as impressions

If you scrape 500 prompts, that does not mean 500 people saw those answers. Your scraper measures a controlled sample of answer states. Use first-party platform impression reports when they are available.

Post-Citation Engagement

Post-Citation Engagement measures whether people who arrived through an AI citation took a meaningful action.

[ \text{Post-Citation Engagement}

\frac{\text{Engaged AI sessions}} {\text{AI-attributed sessions}} ]

For Google Analytics 4, an engaged session lasts more than 10 seconds, includes a key event, or includes at least two page or screen views. Google Analytics engagement-rate documentation (support.google.com)

Also report:

Post-Citation Engagement Time
= median engaged seconds per AI session

Post-Citation Conversion Rate
= AI sessions with a key event / AI-attributed sessions

Post-Citation Revenue per Session
= revenue from AI sessions / AI-attributed sessions

Median engagement time is usually more stable than average engagement time because a small number of very long sessions can distort the average.

Additional Useful Metrics

AI Crawler Success Rate

[ \text{Crawler Success Rate}

\frac{\text{AI crawler requests returning 2xx or 3xx}} {\text{AI crawler requests}} ]

Break this down by bot and path. A high citation decline combined with a rising crawler error rate may indicate a hosting, firewall, or robots configuration problem.

Citation-to-Click Rate

For controlled answer-panel observations:

[ \text{Observed Citation-to-Click Rate}

\frac{\text{Confirmed short-link clicks}} {\text{Observed target citations}} ]

This is not a true user click-through rate unless the denominator represents actual user impressions. Label it clearly as an observed operational metric.

Citation Freshness

Measure how often the cited URL reflects the current page version.

Citation Freshness
= cited pages with current content hash / cited pages

This is useful for documentation, pricing, product, and policy pages that change frequently.

5. Server Log Analysis

Configure Structured Access Logs

Structured logs are easier to analyze than traditional space-separated logs.

Example Nginx configuration:

log_format ai_json escape=json
'{'
  '"ts":"$time_iso8601",'
  '"remote_addr":"$remote_addr",'
  '"request":"$request",'
  '"status":$status,'
  '"bytes":$body_bytes_sent,'
  '"referer":"$http_referer",'
  '"user_agent":"$http_user_agent",'
  '"host":"$host"'
'}';

access_log /var/log/nginx/ai-access.jsonl ai_json;

Store the log files in a restricted location. Do not send raw IP addresses to a general analytics table unless you have a clear operational reason.

Identify AI Request Types

At minimum, separate:

OpenAI:
- OAI-SearchBot
- ChatGPT-User
- GPTBot

Anthropic:
- Claude-SearchBot
- Claude-User
- ClaudeBot

Perplexity:
- PerplexityBot
- Perplexity-User

Search engines:
- Googlebot
- Bingbot

Google’s AI Search features generally use existing Google Search crawling controls rather than a separate universal “AI Overview bot.” Google states that Googlebot controls access for Search, including AI features. Google AI features and websites (developers.google.com)

User-agent strings can be forged. For operational decisions, combine user-agent matching with provider-published IP ranges or verified reverse DNS where available. OpenAI and Perplexity publish IP information for their relevant crawlers and fetchers. (openai.com)

Reproducible Log Summarizer

Save the following as ai_log_summary.py.

#!/usr/bin/env python3

import csv
import hashlib
import hmac
import json
import os
import re
import sys
from collections import Counter, defaultdict
from datetime import datetime
from urllib.parse import urlsplit

BOT_PATTERNS = {
    "OAI-SearchBot": ("openai", re.compile(r"\bOAI-SearchBot\b", re.I)),
    "ChatGPT-User": ("openai_user", re.compile(r"\bChatGPT-User\b", re.I)),
    "GPTBot": ("openai_training", re.compile(r"\bGPTBot\b", re.I)),
    "Claude-SearchBot": ("anthropic", re.compile(r"\bClaude-SearchBot\b", re.I)),
    "Claude-User": ("anthropic_user", re.compile(r"\bClaude-User\b", re.I)),
    "ClaudeBot": ("anthropic_training", re.compile(r"\bClaudeBot\b", re.I)),
    "PerplexityBot": ("perplexity", re.compile(r"\bPerplexityBot\b", re.I)),
    "Perplexity-User": ("perplexity_user", re.compile(r"\bPerplexity-User\b", re.I)),
    "Googlebot": ("google_search", re.compile(r"\bGooglebot\b", re.I)),
    "Bingbot": ("bing_search", re.compile(r"\bbingbot\b", re.I)),
}

SECRET = os.environ.get("HMAC_SECRET", "replace-this-in-production").encode()


def hmac_ip(ip):
    return hmac.new(SECRET, ip.encode(), hashlib.sha256).hexdigest()[:16]


def classify_bot(user_agent):
    for bot_name, (role, pattern) in BOT_PATTERNS.items():
        if pattern.search(user_agent or ""):
            return bot_name, role
    return None, None


def parse_path(request_line):
    try:
        parts = request_line.split()
        return urlsplit(parts[1]).path
    except Exception:
        return "/"


def main(input_path):
    totals = defaultdict(lambda: {
        "requests": 0,
        "success": 0,
        "errors": 0,
        "ips": set(),
        "paths": Counter(),
    })

    with open(input_path, "r", encoding="utf-8") as source:
        for line in source:
            try:
                record = json.loads(line)
            except json.JSONDecodeError:
                continue

            bot_name, bot_role = classify_bot(record.get("user_agent", ""))
            if not bot_name:
                continue

            timestamp = record.get("ts", "")
            date = datetime.fromisoformat(
                timestamp.replace("Z", "+00:00")
            ).date().isoformat()

            status = int(record.get("status", 0))
            path = parse_path(record.get("request", ""))
            key = (date, bot_name, bot_role)

            totals[key]["requests"] += 1
            totals[key]["ips"].add(hmac_ip(record.get("remote_addr", "")))
            totals[key]["paths"][path] += 1

            if 200 <= status < 400:
                totals[key]["success"] += 1
            else:
                totals[key]["errors"] += 1

    writer = csv.writer(sys.stdout)
    writer.writerow([
        "date",
        "bot_name",
        "bot_role",
        "requests",
        "successful_requests",
        "error_requests",
        "unique_ip_count",
        "top_paths",
    ])

    for (date, bot_name, bot_role), values in sorted(totals.items()):
        top_paths = values["paths"].most_common(10)
        writer.writerow([
            date,
            bot_name,
            bot_role,
            values["requests"],
            values["success"],
            values["errors"],
            len(values["ips"]),
            json.dumps(top_paths),
        ])


if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python ai_log_summary.py /path/to/access.jsonl")
        sys.exit(1)

    main(sys.argv[1])

Run it with:

HMAC_SECRET='a-long-random-secret' \
python ai_log_summary.py /var/log/nginx/ai-access.jsonl \
> data/ai_crawler_daily.csv

This produces operational evidence about retrieval activity, not human traffic.

6. Custom Link Shorteners for AI-Facing Summaries

A first-party link shortener provides a controlled measurement point when you publish summaries or source materials that may be referenced by AI systems.

Use a domain you control, such as:

https://go.example.com/ai-docs

The destination should include campaign parameters such as:

utm_source=perplexity
utm_medium=ai_answer
utm_campaign=product_documentation
utm_content=pricing_summary_v3

Google recommends campaign parameters for identifying traffic sources and campaigns. Google Analytics campaign URL documentation (support.google.com)

Important limitations

A short link is a calibration device, not a universal solution:

The assistant may rewrite the destination URL.
The assistant may cite the canonical URL instead.
The link may be displayed but never clicked.
A browser or privacy tool may remove the referrer.
Some systems may fetch the destination server-side.
A short link does not measure users who remember the brand and return later.

Do not use a shortener to hide the destination, bypass safety systems, or manipulate citations. Use a clear redirect to a relevant, canonical page.

Minimal First-Party Shortener

The following example uses FastAPI and SQLite. Protect the link-creation endpoint with authentication before using it in production.

# shortener.py
import hashlib
import hmac
import os
import secrets
import sqlite3
from datetime import datetime, timezone
from urllib.parse import parse_qsl, urlencode, urlsplit, urlunsplit

from fastapi import FastAPI, HTTPException, Request
from fastapi.responses import RedirectResponse
from pydantic import BaseModel

DB_PATH = os.environ.get("SHORTENER_DB", "shortener.db")
HMAC_SECRET = os.environ.get(
    "HMAC_SECRET", "replace-this-in-production"
).encode()

ALLOWED_HOST = "www.example.com"

app = FastAPI()


def db():
    connection = sqlite3.connect(DB_PATH)
    connection.row_factory = sqlite3.Row
    return connection


def init_db():
    connection = db()
    connection.execute("""
        CREATE TABLE IF NOT EXISTS links (
            slug TEXT PRIMARY KEY,
            destination TEXT NOT NULL,
            surface TEXT NOT NULL,
            topic TEXT,
            prompt_id TEXT,
            asset_id TEXT,
            created_at TEXT NOT NULL
        )
    """)
    connection.execute("""
        CREATE TABLE IF NOT EXISTS clicks (
            id INTEGER PRIMARY KEY AUTOINCREMENT,
            occurred_at TEXT NOT NULL,
            slug TEXT NOT NULL,
            surface TEXT,
            topic TEXT,
            prompt_id TEXT,
            referrer_host TEXT,
            user_agent TEXT,
            ip_hmac TEXT
        )
    """)
    connection.commit()
    connection.close()


def append_query(url, params):
    parsed = urlsplit(url)
    query = dict(parse_qsl(parsed.query, keep_blank_values=True))
    query.update(params)
    return urlunsplit((
        parsed.scheme,
        parsed.netloc,
        parsed.path,
        urlencode(query),
        parsed.fragment,
    ))


def hash_ip(ip):
    return hmac.new(
        HMAC_SECRET,
        (ip or "").encode(),
        hashlib.sha256
    ).hexdigest()[:16]


class LinkRequest(BaseModel):
    destination_path: str
    surface: str
    topic: str | None = None
    prompt_id: str | None = None
    asset_id: str | None = None


@app.on_event("startup")
def startup():
    init_db()


@app.post("/links")
def create_link(payload: LinkRequest):
    destination = f"https://{ALLOWED_HOST}{payload.destination_path}"

    if not payload.destination_path.startswith("/"):
        raise HTTPException(400, "destination_path must be a site-relative path")

    slug = secrets.token_urlsafe(7).replace("_", "-").replace("/", "-")

    target = append_query(destination, {
        "utm_source": payload.surface,
        "utm_medium": "ai_answer",
        "utm_campaign": "citation_measurement",
        "utm_content": payload.asset_id or "unspecified",
    })

    connection = db()
    connection.execute("""
        INSERT INTO links
        (slug, destination, surface, topic, prompt_id, asset_id, created_at)
        VALUES (?, ?, ?, ?, ?, ?, ?)
    """, (
        slug,
        target,
        payload.surface,
        payload.topic,
        payload.prompt_id,
        payload.asset_id,
        datetime.now(timezone.utc).isoformat(),
    ))
    connection.commit()
    connection.close()

    return {"slug": slug, "url": f"https://go.example.com/{slug}"}


@app.get("/{slug}")
def redirect(slug: str, request: Request):
    connection = db()
    row = connection.execute(
        "SELECT * FROM links WHERE slug = ?", (slug,)
    ).fetchone()

    if not row:
        connection.close()
        raise HTTPException(404, "Unknown link")

    referrer = request.headers.get("referer", "")
    referrer_host = urlsplit(referrer).hostname or ""

    connection.execute("""
        INSERT INTO clicks
        (occurred_at, slug, surface, topic, prompt_id,
         referrer_host, user_agent, ip_hmac)
        VALUES (?, ?, ?, ?, ?, ?, ?, ?)
    """, (
        datetime.now(timezone.utc).isoformat(),
        slug,
        row["surface"],
        row["topic"],
        row["prompt_id"],
        referrer_host,
        request.headers.get("user-agent", "")[:300],
        hash_ip(request.client.host if request.client else ""),
    ))
    connection.commit()
    connection.close()

    return RedirectResponse(row["destination"], status_code=302)

Run it with:

pip install fastapi uvicorn
uvicorn shortener:app --host 0.0.0.0 --port 8080

Use a 302 redirect for measurement links so browsers and intermediate systems are less likely to cache the redirect permanently.

7. Scheduled Scraping of AI Answer Panels

Panel scraping is useful for answering questions that first-party analytics may not expose:

Which competing sources are cited?
Which page is cited for a specific prompt?
Does the answer change by country or device?
Does the citation appear in the answer, the source list, or both?
How stable is the answer over time?
Does the assistant cite a short link or the canonical URL?

However, scraping must be done carefully.

Rules for responsible collection

Prefer an official API, licensed search data provider, or first-party report.
Review the provider’s terms before automating requests.
Respect robots instructions where applicable.
Do not bypass CAPTCHAs, login barriers, rate limits, or bot protections.
Use a small, fixed prompt set.
Run at low frequency.
Save the response and timestamp so results can be audited.
Stop when a provider returns a challenge or access-denied response.

The Robots Exclusion Protocol is a crawler preference mechanism, not a security system. It does not authorize access to protected content. RFC 9309: Robots Exclusion Protocol (rfc-editor.org)

Build a Prompt Manifest

prompt_id,topic,query,locale,surface,target_domain
pm_001,project-management,"best project management software for remote teams",en-US,google,example.com
pm_001,project-management,"best project management software for remote teams",en-US,bing,example.com
pm_002,analytics,"how to measure AI referral traffic",en-US,google,example.com

Keep prompts stable. If prompts change, trend comparisons become less reliable.

Also store:

country
language
device type
browser version
prompt version
content version
collection time in UTC

Google notes that AI Overview and AI Mode responses may vary because they use different systems and may use related searches or “query fan-out.” Google AI features and website documentation (developers.google.com)

Playwright Collection Script

The following script is a general-purpose research harness. It records the rendered page, screenshot, page links, and answer text. It intentionally does not bypass challenges or attempt to defeat access controls.

# scrape_panels.py
import asyncio
import csv
import hashlib
import json
import os
import re
from datetime import datetime, timezone
from pathlib import Path
from urllib.parse import quote_plus, urljoin, urlsplit

from playwright.async_api import async_playwright

DATA_DIR = Path("data/panels")
DATA_DIR.mkdir(parents=True, exist_ok=True)

SURFACE_URLS = {
    "google": "https://www.google.com/search?q={query}&hl=en&gl=us",
    "bing": "https://www.bing.com/search?q={query}&setlang=en-us",
}

PANEL_MARKERS = {
    "google": ["AI Overview", "AI Mode"],
    "bing": ["Copilot", "Generative search"],
}

BLOCK_MARKERS = [
    "captcha",
    "unusual traffic",
    "verify you are human",
    "access denied",
]


def clean_text(value):
    return re.sub(r"\s+", " ", value or "").strip()


def host_matches(url, target_domain):
    host = (urlsplit(url).hostname or "").lower()
    target = target_domain.lower().lstrip("www.")
    return host == target or host.endswith("." + target)


def page_hash(text):
    normalized = clean_text(text).encode("utf-8")
    return hashlib.sha256(normalized).hexdigest()


async def collect_one(page, row):
    surface = row["surface"]
    query = row["query"]
    target_domain = row["target_domain"]

    search_url = SURFACE_URLS[surface].format(
        query=quote_plus(query)
    )

    await page.goto(
        search_url,
        wait_until="domcontentloaded",
        timeout=45_000,
    )
    await page.wait_for_timeout(2500)

    text = await page.locator("body").inner_text()
    lowered = text.lower()

    if any(marker in lowered for marker in BLOCK_MARKERS):
        return {
            **row,
            "observed_at": datetime.now(timezone.utc).isoformat(),
            "status": "blocked_or_challenged",
            "panel_present": False,
            "links": [],
        }

    links = await page.locator("a").evaluate_all("""
        anchors => anchors.map(a => ({
            href: a.href || "",
            text: (a.innerText || "").trim()
        })).filter(x => x.href)
    """)

    panel_present = any(
        marker.lower() in lowered
        for marker in PANEL_MARKERS.get(surface, [])
    )

    observed_at = datetime.now(timezone.utc)
    run_id = hashlib.sha256(
        f"{surface}|{row['prompt_id']}|{observed_at.isoformat()}".encode()
    ).hexdigest()[:20]

    html_path = DATA_DIR / f"{run_id}.html"
    screenshot_path = DATA_DIR / f"{run_id}.png"

    await page.screenshot(path=str(screenshot_path), full_page=True)
    html_path.write_text(await page.content(), encoding="utf-8")

    output_links = []
    for position, link in enumerate(links, start=1):
        output_links.append({
            "href": urljoin(search_url, link["href"]),
            "text": clean_text(link["text"])[:300],
            "position": position,
            "target_cited": host_matches(
                link["href"],
                target_domain
            ),
        })

    return {
        **row,
        "run_id": run_id,
        "observed_at": observed_at.isoformat(),
        "status": "ok",
        "panel_present": panel_present,
        "answer_hash": page_hash(text),
        "answer_text": clean_text(text)[:50_000],
        "html_path": str(html_path),
        "screenshot_path": str(screenshot_path),
        "link_scope": "page_fallback",
        "links": output_links,
    }


async def main(manifest_path):
    rows = []
    with open(manifest_path, newline="", encoding="utf-8") as source:
        rows = list(csv.DictReader(source))

    output_path = DATA_DIR / (
        datetime.now(timezone.utc).strftime("%Y-%m-%d") + ".jsonl"
    )

    async with async_playwright() as playwright:
        browser = await playwright.chromium.launch(headless=True)
        context = await browser.new_context(
            locale="en-US",
            timezone_id="UTC",
        )
        page = await context.new_page()

        with open(output_path, "w", encoding="utf-8") as output:
            for row in rows:
                try:
                    result = await collect_one(page, row)
                except Exception as exc:
                    result = {
                        **row,
                        "observed_at": datetime.now(timezone.utc).isoformat(),
                        "status": f"error:{type(exc).__name__}",
                        "panel_present": False,
                        "links": [],
                    }

                output.write(json.dumps(result) + "\n")
                output.flush()

                # Keep request volume low.
                await asyncio.sleep(8)

        await browser.close()


if __name__ == "__main__":
    asyncio.run(main("prompts.csv"))

Install and run:

pip install playwright
playwright install chromium
python scrape_panels.py

Schedule it with a lock so two runs cannot overlap:

15 3 * * * flock -n /tmp/ai-panel.lock \
  /usr/bin/python3 /opt/ai-measure/scrape_panels.py

Improve accuracy with surface adapters

The fallback script records page-level links. That is useful for discovery but may include ordinary search-result links. For production use:

Use a licensed answer-panel API when available.
Create a separate selector adapter for each surface.
Store the selector version.
Store raw HTML and a screenshot.
Mark every record with link_scope = "panel_selector" or link_scope = "page_fallback".

Only calculate Citation Share from panel_selector data when you can reliably identify the panel. Keep fallback observations in a separate exploratory report.

8. Google Analytics and BigQuery Integration

Google Analytics 4 can export raw event data to BigQuery. Daily exports provide stable tables such as events_YYYYMMDD; streaming exports are useful for near-real-time monitoring but may be incomplete until the daily table is finalized. Google Analytics BigQuery Export (support.google.com)

A simplified session query might look like this:

WITH page_events AS (
  SELECT
    event_date,
    user_pseudo_id,

    (
      SELECT value.int_value
      FROM UNNEST(event_params)
      WHERE key = 'ga_session_id'
    ) AS session_id,

    (
      SELECT value.string_value
      FROM UNNEST(event_params)
      WHERE key = 'page_location'
    ) AS page_location,

    (
      SELECT value.string_value
      FROM UNNEST(event_params)
      WHERE key = 'session_engaged'
    ) AS session_engaged,

    event_name
  FROM `PROJECT.analytics_PROPERTY.events_*`
  WHERE _TABLE_SUFFIX BETWEEN '20260901' AND '20260921'
),

session_rollup AS (
  SELECT
    event_date,
    user_pseudo_id,
    session_id,

    MAX(
      CASE
        WHEN REGEXP_CONTAINS(
          LOWER(page_location),
          r'[?&]utm_medium=ai_answer'
        )
        THEN 1 ELSE 0
      END
    ) AS ai_session,

    MAX(
      CASE
        WHEN session_engaged = '1'
        THEN 1 ELSE 0
      END
    ) AS engaged_session,

    MAX(
      CASE
        WHEN event_name IN ('purchase', 'generate_lead', 'sign_up')
        THEN 1 ELSE 0
      END
    ) AS key_event

  FROM page_events
  GROUP BY
    event_date,
    user_pseudo_id,
    session_id
)

SELECT
  event_date,
  COUNTIF(ai_session = 1) AS ai_sessions,
  COUNTIF(ai_session = 1 AND engaged_session = 1)
    AS engaged_ai_sessions,
  COUNTIF(ai_session = 1 AND key_event = 1)
    AS ai_key_events,
  SAFE_DIVIDE(
    COUNTIF(ai_session = 1 AND engaged_session = 1),
    COUNTIF(ai_session = 1)
  ) AS post_citation_engagement_rate
FROM session_rollup
GROUP BY event_date
ORDER BY event_date;

For a mature implementation, add surface, topic, and prompt_id to the campaign structure or join them from the shortener click table.

Do not create a high-cardinality analytics dimension for every individual user or session. Google warns that excessive high-cardinality custom dimensions can reduce reporting quality. Google Analytics custom dimensions and metrics (support.google.com)

9. Dashboard Design

A practical dashboard should contain five views.

Executive Summary

Display:

Citation Share by Topic
Citation Coverage by Topic
Confirmed Assistant Referral Rate
Expanded Assistant Referral Rate
Post-Citation Engagement
AI crawler success rate
AI-attributed key events or revenue

Citation Share Trend

Line chart:

Date × Topic × Surface

Show both:

target-domain citation occurrences
all observed citation occurrences

Assistant Traffic Funnel

Observed citations
        ↓
Controlled-link clicks
        ↓
AI-attributed sessions
        ↓
Engaged AI sessions
        ↓
Key events

Every stage should display its attribution confidence.

Crawler Health

Table by bot:

bot_name
requests
successful_requests
error_rate
unique_ip_count
top_paths

Alert when:

error rate exceeds 2 percent
a bot suddenly disappears
a previously accessible content section returns 403 or 429
crawler requests concentrate on an unintended private path

Content Opportunity Table

Rank pages and topics by:

high citation share + low referral rate
high impressions + low citation coverage
high engagement + low citation visibility
high crawler errors + high business importance

These combinations usually produce better decisions than looking only at traffic volume.

Minimal Streamlit Dashboard

Save as dashboard.py.

import glob
import json
from pathlib import Path

import pandas as pd
import streamlit as st

st.set_page_config(
    page_title="AI Citation Impact",
    layout="wide",
)


@st.cache_data(ttl=300)
def load_panel_rows():
    rows = []

    for filename in glob.glob("data/panels/*.jsonl"):
        with open(filename, encoding="utf-8") as source:
            for line in source:
                record = json.loads(line)

                run_id = record.get("run_id")
                for link in record.get("links", []):
                    hostname = link.get("href", "").split("/")[2]
                    rows.append({
                        "run_id": run_id,
                        "observed_at": record.get("observed_at"),
                        "surface": record.get("surface"),
                        "topic": record.get("topic"),
                        "prompt_id": record.get("prompt_id"),
                        "panel_present": record.get("panel_present"),
                        "target_domain": record.get("target_domain"),
                        "cited_domain": hostname,
                        "target_cited": link.get("target_cited", False),
                        "position": link.get("position"),
                        "link_scope": record.get("link_scope"),
                    })

    return pd.DataFrame(rows)


panels = load_panel_rows()

st.title("AI Citation Impact Dashboard")

if panels.empty:
    st.warning("No panel observations found.")
    st.stop()

panels["date"] = pd.to_datetime(
    panels["observed_at"], errors="coerce"
).dt.date

citation_share = (
    panels.groupby("topic")
    .apply(
        lambda group: group["target_cited"].sum()
        / max(len(group), 1)
    )
    .reset_index(name="citation_share")
)

coverage = (
    panels.groupby(["topic", "run_id"], dropna=False)
    .agg(
        panel_present=("panel_present", "max"),
        target_cited=("target_cited", "max"),
    )
    .reset_index()
)

coverage = (
    coverage[coverage["panel_present"] == True]
    .groupby("topic")
    .apply(lambda group: group["target_cited"].mean())
    .reset_index(name="citation_coverage")
)

col1, col2 = st.columns(2)

with col1:
    st.subheader("Citation Share by Topic")
    st.dataframe(citation_share, use_container_width=True)
    st.bar_chart(
        citation_share.set_index("topic")["citation_share"]
    )

with col2:
    st.subheader("Citation Coverage by Topic")
    st.dataframe(coverage, use_container_width=True)
    st.bar_chart(
        coverage.set_index("topic")["citation_coverage"]
    )

st.subheader("Citation Position")
st.dataframe(
    panels.groupby(["topic", "surface"], dropna=False)
    .agg(
        citations=("target_cited", "sum"),
        median_position=("position", "median"),
        observations=("run_id", "nunique"),
    )
    .reset_index(),
    use_container_width=True,
)

st.subheader("Daily Observations")
daily = (
    panels.groupby(["date", "surface"], dropna=False)
    .agg(
        citations=("target_cited", "sum"),
        observed_links=("cited_domain", "count"),
    )
    .reset_index()
)

st.line_chart(
    daily.pivot(
        index="date",
        columns="surface",
        values="citations"
    ).fillna(0)
)

Run it with:

pip install streamlit pandas
streamlit run dashboard.py

For production, load normalized tables into DuckDB, BigQuery, Snowflake, or PostgreSQL instead of reading raw files directly.

10. Quality Controls

A measurement system can produce precise-looking but misleading numbers. Add these controls.

Separate bots from people

Never count OAI-SearchBot, GPTBot, ClaudeBot, or PerplexityBot as human sessions. These are crawler or retrieval signals.

Preserve raw evidence

For every panel run, retain:

raw HTML
screenshot
answer text
timestamp
browser and collector version
prompt and locale
answer hash

This makes it possible to explain why a metric changed.

Track panel absence

A missing AI panel is not the same as an answer without your citation.

Use:

panel_not_present
panel_present_no_target
panel_present_target_cited
blocked_or_challenged
collection_error

Control sampling bias

A scheduled scrape is affected by:

location
device
language
time
personalization
browser state
prompt wording

Use a fixed core set of prompts and a rotating set of exploratory prompts. Keep the two groups separate in reports.

Avoid false precision

Report confidence intervals or minimum sample sizes for small topic groups. A topic with three observations should not be compared directly with a topic containing 300 observations.

Compare like with like

When comparing surfaces, keep constant:

prompt
locale
country
device
collection hour
target-domain rules
citation parser version

11. Recommended Rollout Plan

Week 1: Instrumentation

Configure structured server logs.
Create the bot registry.
Create the shortener domain.
Add campaign naming rules.
Define the topic taxonomy.
Enable Google Analytics 4 BigQuery export.

Week 2: Controlled links

Publish five to ten tracked summaries.
Create links for different topics and content versions.
Validate redirect logs against analytics sessions.
Confirm that no personal information enters URLs or analytics parameters.

Week 3: Panel observations

Create a prompt manifest.
Start with two surfaces and 20 to 50 stable prompts.
Run once per day.
Save screenshots and HTML.
Measure collection failures separately from citation absence.

Week 4: Dashboard and governance

Build topic and surface dashboards.
Add crawler health alerts.
Define confidence rules.
Compare first-party reports with panel observations.
Document retention, access, and deletion policies.

Conclusion

AI citation measurement should be built as a multi-signal attribution system, not as a single referral report.

The most reliable framework combines:

Google and Bing first-party visibility reports
server logs for crawler and fetcher activity
first-party short links for controlled click measurement
scheduled, compliant answer-panel observations
analytics and conversion data for downstream value

The three most important metrics are:

Citation Share by Topic — how much of the observed source space belongs to your domain.
Assistant Referral Rate — how much human traffic can be attributed to AI surfaces.
Post-Citation Engagement — whether AI-referred visitors take meaningful actions.

The system becomes useful when every number includes its surface, time period, denominator, attribution method, and confidence level. That discipline prevents crawler hits from being mistaken for users, scraped answers from being mistaken for impressions, and citations from being mistaken for business results.