Build Your Own AI Visibility Tracker

With all four in one view the questions stop being guesses. Which content type is actually earning AI citations: answer pages, long-form guides, data pages, location pages? Which topics are already winning answers, so you expand them? Which pages absorb production budget and return nothing? Those are content strategy decisions, and right now most agencies make them from whichever console they happened to open.

There is a second reason to build this. When an assistant recommends you to somebody ready to buy, standard analytics records nothing at all: no impression, no click, no referrer. A page that is winning looks identical to a page that is failing. This build tells the two apart. Below is the whole thing, every script and cron entry, plus the seven traps worth knowing before you start. Stage one needs nothing installed.

Why your analytics cannot see this

When somebody asks an AI assistant about Medicare in your county, the assistant reads pages, forms an answer, and cites sources. That is a recommendation. Three things then happen that break normal tracking:

• The buyer often does not click at all. They have the answer.
• When they do arrive, many assistants pass no referrer, so it lands as direct.
• When they arrive later by searching your name, it lands as branded search.

So the channel that produced the outcome is invisible, and whichever channel touched last takes the credit. If you have watched direct traffic climb with no explanation, this is a likely part of it.

What you are building

Four collectors, one dashboard, and a reconciler. Each collector answers a different question, and none of them is sufficient alone. The reconciler in stage 7 is what turns four disconnected feeds into one surface you can decide from, and it is the stage most people skip.

Everything lands as append-only JSONL on one small box. No database. Each collector writes one line per site per day, which means a bad rule can be corrected retroactively against history you already hold. That property matters more than it sounds, and stage 5 explains why.

The payoff is in how you slice it. Group citations and clicks by URL pattern and you can see which content type is earning answers on your own site: whether /answers/ pages out-cite long-form guides, whether data pages out-cite both, whether location pages earn anything at all. Group by grounding query and you get the topics where you already hold share and should go deeper. That is a content plan built on your own performance data, and no single console will hand it to you.

Why this build does not use a standard pixel

There is no Google Analytics tag and no Microsoft Clarity anywhere in this build. That is deliberate, and on Medicare or ACA pages it is not optional.

Clarity records sessions. It captures what a visitor sees and types, which on a quoting flow means date of birth, medications and conditions. Microsoft does not offer a BAA for Clarity. Field masking reduces the exposure, but session replay is structurally the wrong tool on health pages.

GA4 reports to Google. Every hit sends the page URL, the referrer and a persistent client identifier. Google will not sign a BAA covering GA4. When the URL or the event parameters carry health context, that combination is the exact fact pattern behind the hospital pixel litigation.

So stage 4 is a first-party beacon. Your endpoint, your storage, nothing leaving your infrastructure by default. If you still want GA4 numbers, you forward them server-side with the health context stripped and a pseudonymous id, so Google receives counts rather than people.

Scope it correctly. This applies to Medicare, ACA and health. On life, annuities and property and casualty, a client-side GA4 tag is fine. Agencies over-apply this and end up blind on business where they never needed to be. None of this is legal advice; run your own setup past your counsel.

What you need before you start

You do not need to know how to code. You need to be able to copy a block of text, paste it, and read what comes back. Every stage below gives you the exact thing to paste and the exact thing you should see when it worked.

• Your sites verified in Bing Webmaster Tools. If they are already in Search Console, importing takes about two minutes.
• A Google account with access to your Search Console. Stage 2 walks through the rest.
• A computer you use most days. Stages 1 to 6 run fine on a laptop. Stage 1 needs nothing installed at all.

You do not need a server to start. Everything here runs on your own machine. Once it works and you want the collectors running while your laptop is shut, you move stages 2 to 5 onto any cheap always-on box, and the schedule section covers that. Stage 6 has to stay on a machine you log into, for a reason explained there.

One decision worth making now, because it is annoying to change later: pick a short nickname for each site (acme, beta) and use it everywhere. Every part of this reads the same brands.json file. The moment two scripts keep their own copy of the site list, they drift apart, and you find out months later when a site you added is quietly missing from one of them.

Setting up Claude Code

Claude Code is Anthropic's coding assistant that runs in your terminal. You describe what you want in plain English and it writes the files, runs them, reads the errors and fixes them. That is why this guide is a list of things to say rather than a list of things to type.

The terminal is the app called Terminal on a Mac or PowerShell on Windows. Open it, then follow the install instructions at docs.claude.com. When it is installed, make a folder for this project and start Claude Code inside it.

Then paste this as your first message:

I want to build a small tool that tracks how often AI assistants
and search engines cite my website. I have never written code.

Start by setting up a project folder for me:

1. Make a folder called ai-visibility and set it up as a Node project.
2. Check my Node version is 18 or newer, and tell me how to fix it if not.
3. Create a file called brands.json containing my sites, in this shape:
   { "shortid": "mydomain.com" }
   Ask me for my domains and pick sensible short ids.
4. Create an empty .env file and add it to .gitignore, because API keys go
   there and must never be committed.

Explain what each file is for in plain English as you go. Do not assume I know
what any of this means.

You should see: a new folder with a package.json, a brands.json listing your sites, and an empty .env. Claude Code should have explained what each one is for.

Two habits worth having from the start. If something does not work, paste the error back and say "this failed, here is the output" rather than trying to interpret it yourself. And if you do not understand a file it just wrote, ask it to explain that file line by line. Neither of those is a detour. That is the intended way to use it.

Stage 1: read your AI citations today

Microsoft publishes this and almost nobody uses it. Bing Webmaster Tools has an AI Performance report: how often Copilot and Bing's AI answers cited your pages, the grounding queries that triggered each citation, and your citation share against everyone else answering that question. Verify the site, then open the report.

The UI shows one site at a time, which gets old immediately. It is backed by two internal endpoints that authenticate off your signed-in session, so you can read every site at once from the browser console. Open the console on any bing.com page while signed in:

// Paste into the console on bing.com while signed into Webmaster Tools.
// Prints citation totals for every site you own, in one pass.
const tok = (await (await fetch("/webmasters/auth/token",
  { credentials: "include" })).text()).trim();

// A signed-out session still returns a token-shaped response, so never treat
// a token as proof of sign-in. The DATA calls are what actually reject.
if (!/^[a-f0-9]{16,}$/i.test(tok)) throw new Error("signed out");

const post = async (path, siteUrl) => {
  const r = await fetch(path, {
    method: "POST",
    headers: { "Content-Type": "application/json", "X-CSRF-Token": tok },
    credentials: "include",
    body: JSON.stringify({ siteUrl }),
  });
  return [r.status, await r.text()];
};

for (const domain of ["example.com", "example.org"]) {
  const [status, body] = await post(
    "/webmasters/api/aiperformance/citationstats",
    "https://" + domain + "/"
  );
  if (status !== 200) { console.log(domain, "HTTP", status); continue; }
  const stats = JSON.parse(body).CitationStats || [];
  const total = stats.reduce((a, x) => a + x.Citations, 0);
  console.log(domain, total, "citations over", stats.length, "days");
}

You should see one line per site, with a citation count and the number of days Bing holds data for. A zero is a real answer, not a failure. If you get signed out, you are not signed into Bing in that tab. If one site prints HTTP 403, you are signed into an account that does not own that site, which matters more than it sounds and comes back in stage 6.

Nothing is installed yet and nothing has changed on your site. This stage is read-only. If you would rather not paste code you cannot read, paste it into Claude Code first and ask it to explain what the snippet does before you run it.

Three things about those endpoints, each of which cost me time:

• The auth token is not proof you are signed in. A signed-out session still hands you a token-shaped response. Only the data calls reject. That is why the check above tests the token's shape, and the code still treats the first data call as the real verdict.
• Date-range parameters are accepted and ignored. Each call returns the full history Bing holds for that property, so there is no paging and no windowing to write.
• HTTP 404 from the queries endpoint is not an error. It means Bing holds no rows for that site yet. Sites with zero citations return it normally.

Now the part actually worth reading. The counts tell you how loud you are. The grounding queries tell you what you are known for:

// The counts tell you how loud you are. This tells you what you are known for.
const [status, body] = await post(
  "/webmasters/api/aiperformance/searchqueries/stats",
  "https://example.com/"
);

// 404 NoDataFound means Bing holds no rows for this site yet. It is not an
// error, and sites with zero citations return it normally.
const queries = status === 200 ? JSON.parse(body).Queries || [] : [];

console.table(queries
  .sort((a, b) => b.Citations - a.Citations)
  .slice(0, 20)
  .map(q => ({
    query:     q.GroundingQuery,
    intent:    (q.Intent || []).join(", "),
    topic:     [...(q.Topics || []), ...(q.TopicClusters || [])].join(", "),
    citations: q.Citations,
    share:     Math.round(q.CitationRate * 1000) / 10 + "%",
  })));

On one property I run, the single biggest grounding query was a long, awkward question about group life participation minimums. It was not a keyword anyone would have chosen and it was not in any content plan, and it turned out to be the thing that site was best known for. Read your list before you write another page.

Stage 2: Search Console on a schedule

A service account pulls Search Console without anybody logging in. In the Google Cloud console: create a project, enable the Search Console API, create a service account, and download a JSON key. Then in Search Console, add the service account's email as a user with Full permission on every property. Owner is not required. Full is.

Save that key file into your project folder, then paste this:

Write me a Node script that pulls Google Search Console data for
every site in ./brands.json once a day, and appends the results to a JSONL file.

Setup I already have:
- A Google Cloud service account key file at ./gsc-service-account.json
- The service account email added with Full permission on each property

Requirements, and these matter:
- No npm packages. Sign the JWT assertion yourself with node:crypto and exchange
  it at https://oauth2.googleapis.com/token for an access token.
  Scope: https://www.googleapis.com/auth/webmasters
- Query searchAnalytics with dimensions ["query"] and rowLimit 500, over a
  90-day window ENDING 3 DAYS AGO. Search Console finalises data on a lag, so
  never request today or the last few days come back empty.
- Write one line per site per day to ./gsc-daily.jsonl with clicks, impressions,
  query count, and the top 25 queries by impressions.
- If one site fails, append the error AS A ROW and keep going. Never skip a site
  silently, or a dead property stays dead and I never find out.
- Give me a second mode that just LISTS the properties the service account can
  actually see, so I can confirm access before debugging missing data.

Run the access check first and show me what it prints.

You should see the access check first: a list of your properties with a permission level next to each. Check your sites are all in that list before going further. If one is missing, it is missing everywhere downstream, and you will otherwise spend an evening debugging an empty report that was never a code problem.

If it fails, paste the error straight back. The two common ones are a key file in the wrong place and the service account not actually added in Search Console, and Claude Code will tell them apart from the error text.

For reference, this is the authentication piece it should produce. No Google client library, because Node can sign the assertion itself:

// gsc-auth.mjs
// Google Search Console via a service-account JWT. No external dependencies:
// Node signs the assertion itself.
import { readFileSync } from "node:fs";
import { createSign } from "node:crypto";

const KEYFILE = process.env.GSC_KEY || "./gsc-service-account.json";
const SCOPE = "https://www.googleapis.com/auth/webmasters";

export function creds() {
  return JSON.parse(readFileSync(KEYFILE, "utf8"));
}

export async function getToken() {
  const c = creds();
  const now = Math.floor(Date.now() / 1000);
  const header = { alg: "RS256", typ: "JWT" };
  const claim = {
    iss: c.client_email,
    scope: SCOPE,
    aud: "https://oauth2.googleapis.com/token",
    exp: now + 3600,
    iat: now,
  };
  const b64 = (o) => Buffer.from(JSON.stringify(o)).toString("base64url");
  const unsigned = `${b64(header)}.${b64(claim)}`;
  const sig = createSign("RSA-SHA256")
    .update(unsigned)
    .sign(c.private_key, "base64url");
  const jwt = `${unsigned}.${sig}`;

  const res = await fetch("https://oauth2.googleapis.com/token", {
    method: "POST",
    headers: { "content-type": "application/x-www-form-urlencoded" },
    body: new URLSearchParams({
      grant_type: "urn:ietf:params:oauth:grant-type:jwt-bearer",
      assertion: jwt,
    }),
  });
  const j = await res.json();
  if (!j.access_token) throw new Error("token failed: " + JSON.stringify(j).slice(0, 300));
  return j.access_token;
}

export async function gsc(path, body, token) {
  const t = token || (await getToken());
  const res = await fetch(`https://searchconsole.googleapis.com/${path}`, {
    method: body ? "POST" : "GET",
    headers: { authorization: `Bearer ${t}`, "content-type": "application/json" },
    body: body ? JSON.stringify(body) : undefined,
  });
  const j = await res.json().catch(() => ({}));
  if (!res.ok) throw new Error(`${res.status} ${JSON.stringify(j).slice(0, 300)}`);
  return j;
}

// Run this file directly to list what the service account can actually see.
// If a property is missing here, it is missing everywhere downstream.
if (import.meta.url === `file://${process.argv[1]}`) {
  const sites = await gsc("webmasters/v3/sites");
  const list = sites.siteEntry || [];
  console.log(`service account: ${creds().client_email}`);
  console.log(`accessible properties: ${list.length}`);
  for (const s of list) console.log(`  ${s.permissionLevel.padEnd(18)} ${s.siteUrl}`);
}

And the daily pull:

// gsc-pull.mjs, one row per site per day, appended to gsc-daily.jsonl.
import { appendFileSync, readFileSync } from "node:fs";
import { gsc, getToken } from "./gsc-auth.mjs";

const BRANDS = JSON.parse(readFileSync("./brands.json", "utf8"));
const OUT = "./gsc-daily.jsonl";
const iso = (d) => d.toISOString().slice(0, 10);

// Search Console finalises data on a lag, so never ask for "today".
const end = new Date(Date.now() - 3 * 864e5);
const start = new Date(end - 89 * 864e5);
const token = await getToken();

for (const [id, domain] of Object.entries(BRANDS)) {
  const property = `https://${domain}/`;
  try {
    const q = await gsc(
      `webmasters/v3/sites/${encodeURIComponent(property)}/searchAnalytics/query`,
      {
        startDate: iso(start),
        endDate: iso(end),
        dimensions: ["query"],
        rowLimit: 500,
      },
      token
    );
    const rows = q.rows || [];
    appendFileSync(OUT, JSON.stringify({
      t: iso(new Date()),
      site: id,
      domain,
      window: `${iso(start)} to ${iso(end)}`,
      clicks:      rows.reduce((a, r) => a + r.clicks, 0),
      impressions: rows.reduce((a, r) => a + r.impressions, 0),
      queryCount:  rows.length,
      topQueries: rows
        .sort((a, b) => b.impressions - a.impressions)
        .slice(0, 25)
        .map((r) => ({
          q: r.keys[0],
          impr: r.impressions,
          clicks: r.clicks,
          pos: Math.round(r.position * 10) / 10,
        })),
    }) + "\n");
    console.log(`${id}: ok`);
  } catch (e) {
    // Log the failure as data. A silent skip is how a dead property stays dead.
    appendFileSync(OUT, JSON.stringify({
      t: iso(new Date()), site: id, error: String(e).slice(0, 200),
    }) + "\n");
    console.log(`${id}: ERROR ${String(e).slice(0, 120)}`);
  }
}

The trap that will get you. A Search Console property is either a Domain property or a URL-prefix property, and they are not interchangeable. If your site serves www and you added the bare domain as a URL-prefix property, you will collect nothing, forever, and see no error at all. The API returns a clean empty result. Prefer Domain properties where you can, and normalise before you join anything:

// The GSC property string and your brand key rarely match exactly.
// Normalise before joining, or the data silently never links up.
function byDomain(propertyUrl) {
  return String(propertyUrl)
    .replace(/^sc-domain:/, "")    // Domain properties
    .replace(/^https?:\/\//, "")   // URL-prefix properties
    .replace(/^www\./, "")         // www variants
    .replace(/\/$/, "")            // trailing slash
    .toLowerCase();
}

byDomain("https://www.example.com/"); // -> "example.com"
byDomain("sc-domain:example.com");    // -> "example.com"

Two smaller ones. Search Console finalises data on a lag, so never request today. And the sitemap API's indexed field is deprecated and always returns 0, so do not render it as a coverage number: it reads like a catastrophic failure and means nothing. Real index status comes from the URL Inspection API.

Stage 3: Bing classic search data

Bing's classic data, unlike the AI report, has a proper REST API. Generate a key under Settings, API Access. One key covers every site on the account. Put it in your .env file as BING_WEBMASTER_API_KEY, then paste this:

Now add Bing's classic search data to the same project, as a
second collector writing to ./bing-daily.jsonl.

- My API key is in .env as BING_WEBMASTER_API_KEY. Read it from there, never
  hardcode it.
- Use the JSON endpoint: https://ssl.bing.com/webmaster/api.svc/json/<Endpoint>
  with siteUrl and apikey as query parameters. If you see a notice about the
  Bing API being retired, that applies to the pox/ and soap/ endpoints, NOT the
  json/ one. Do not migrate anything.
- Pull GetRankAndTrafficStats, GetQueryStats and GetPageStats for each site.
- Every response is wrapped in a ".d" property, so unwrap that first.
- Dates come back as /Date(1755734400000)/ rather than ISO. Parse them.
- Also count how many queries are seven words or longer, and store that as
  conversationalQueries. Long sentence-shaped queries are the fingerprint of
  assistant-driven search, and I want to watch that number move.

Run it and show me one site's row.

You should see a row per site with seven days of impressions and clicks, a query count, and a conversationalQueries number. If every site returns null, the key is wrong or was not read from .env. Here is the collector it should produce:

// bing-pull.mjs, Bing's CLASSIC search data, which does have a real REST API.
// The AI citation report does NOT. That is stage 1 and stage 6.
import { readFileSync, appendFileSync } from "node:fs";

const KEY = process.env.BING_WEBMASTER_API_KEY;
if (!KEY) throw new Error("no BING_WEBMASTER_API_KEY");
const BRANDS = JSON.parse(readFileSync("./brands.json", "utf8"));
const OUT = "./bing-daily.jsonl";

// Use the /json/ endpoint. Microsoft is retiring /pox/ and /soap/ only, so the
// retirement banner in the UI does not apply to this call.
const api = async (endpoint, site) => {
  const u = `https://ssl.bing.com/webmaster/api.svc/json/${endpoint}`
    + `?siteUrl=${encodeURIComponent("https://" + site + "/")}&apikey=${KEY}`;
  const r = await fetch(u, { signal: AbortSignal.timeout(30000) });
  if (!r.ok) return null;
  return (await r.json()).d ?? null;   // every response is wrapped in .d
};

// Bing serialises dates as /Date(1755734400000)/
const dstr = (d) => {
  const ms = Number(String(d).match(/\/Date\((\d+)/)?.[1]);
  return Number.isFinite(ms) ? new Date(ms).toISOString().slice(0, 10) : null;
};

const day = new Date().toISOString().slice(0, 10);

for (const [id, site] of Object.entries(BRANDS)) {
  try {
    const [traffic, queries, pages] = await Promise.all([
      api("GetRankAndTrafficStats", site),
      api("GetQueryStats", site),
      api("GetPageStats", site),
    ]);

    const recent = (traffic || [])
      .map((r) => ({ d: dstr(r.Date), impr: r.Impressions, clicks: r.Clicks }))
      .filter((r) => r.d)
      .sort((a, b) => (a.d < b.d ? 1 : -1))
      .slice(0, 7);

    const qs = (queries || []).map((q) => ({
      q: q.Query, impr: q.Impressions, clicks: q.Clicks, pos: q.AvgImpressionPosition,
    }));

    // Long, sentence-shaped queries are the fingerprint of assistant-driven
    // search. Seven words or more is a crude but reliable cut.
    const conversational = qs.filter((q) => (q.q || "").split(/\s+/).length >= 7);

    appendFileSync(OUT, JSON.stringify({
      t: day, site: id, domain: site,
      last7: recent,
      impr7:   recent.reduce((a, r) => a + (r.impr || 0), 0),
      clicks7: recent.reduce((a, r) => a + (r.clicks || 0), 0),
      queryCount: qs.length,
      conversationalQueries: conversational.length,
      topQueries: qs.sort((a, b) => b.impr - a.impr).slice(0, 10),
      topPages: (pages || []).slice(0, 10).map((p) => ({
        url: p.Query || p.Url, impr: p.Impressions, clicks: p.Clicks,
      })),
    }) + "\n");
  } catch (e) {
    appendFileSync(OUT, JSON.stringify({
      t: day, site: id, error: String(e).slice(0, 150),
    }) + "\n");
  }
}

Ignore the retirement banner. Microsoft is retiring the pox/ and soap/ endpoints. The json/ endpoint above is unaffected, and Microsoft's own notice says so explicitly. I nearly rewrote a working integration over that banner.

Two format quirks that are not documented anywhere obvious: every response is wrapped in a .d envelope, and dates come back as /Date(1755734400000)/ rather than ISO. Both are handled above.

The conversationalQueries count is a cheap proxy worth keeping. Queries of seven words or more are sentence-shaped rather than keyword-shaped, and a rising count is usually assistant-driven search arriving before anything else in your stack notices.

Stage 4: first-party visit beacon

A few lines on your site, and a small endpoint on your own domain, record every visit and classify its source. Because it is first-party it does not get blocked the way third-party tags do, and the data is yours.

I need to know which visits actually reach my site and where
they came from, because AI assistants usually pass no referrer and analytics
files them as "direct".

Build me two things:

1. A small script tag I can paste into my site's layout. It must:
   - fire on EVERY visit, not only when the referrer looks like an AI tool
   - send only: my site id, the path, the referrer's HOSTNAME, and any utm_*
     values. Never the full referrer URL, never anything identifying.
   - skip the referrer entirely when it is my own domain, so internal clicks
     are not logged as referrals
   - use navigator.sendBeacon with a plain string body, so there is no CORS
     preflight
   - support an optional consent gate, because on health pages it must not
     fire until the visitor accepts. Consent scripts often load AFTER inline
     scripts, so listen for an event AND re-check on a timer, then give up.

2. A matching endpoint on my server that appends each hit to ./hits.jsonl.
   Respond 204 immediately, before doing any work, so the page never waits.
   Work out whether the visit was ai / organic / social / campaign / direct
   SERVER-SIDE, from the stored hostname, not in the browser. I want to be able
   to fix that rule later and re-run it over history I already collected.

You should see two pieces: a short script tag, and an endpoint. Add the script tag to your site's layout so it appears on every page, then load a page and check that a row appeared in hits.jsonl. If nothing arrives, the usual cause is the consent gate holding it shut, which is correct behaviour until you accept.

The snippet it should produce:

<!-- Visit beacon v2. Put this in your site's base layout, on every page. -->
<script>
(function () {
  var SITE = "yoursite", GATE = "none", sent = false, ticks = 0, timer = null;

  function send() {
    if (sent) return;
    try {
      if (!navigator.sendBeacon) return;

      // Referrer HOSTNAME only. Never the full URL, and never our own host,
      // so internal clicks are not logged as referrals.
      var h = "";
      try {
        if (document.referrer) {
          var u = new URL(document.referrer);
          if (u.hostname && u.hostname !== location.hostname) h = u.hostname;
        }
      } catch (e) {}

      var d = { s: SITE, p: location.pathname.slice(0, 300), r: h };

      try {
        var q = new URLSearchParams(location.search),
            M = { utm_source: "us", utm_medium: "um", utm_campaign: "uc", utm_term: "ut" };
        for (var k in M) { var v = q.get(k); if (v) d[M[k]] = String(v).slice(0, 80); }
      } catch (e) {}

      sent = true;
      if (timer) { clearInterval(timer); timer = null; }

      // Plain string body, so there is no CORS preflight.
      navigator.sendBeacon("https://collector.yourdomain.com/hit", JSON.stringify(d));
    } catch (e) {}
  }

  function consented() {
    try {
      if (GATE === "none")         return true;
      if (GATE === "localstorage") return localStorage.getItem("site-consent") === "accepted";
      if (GATE === "global")       return window.__consent === "accept";
    } catch (e) {}
    return false;
  }

  // Consent modules are often bundled and can initialise AFTER an inline
  // script, so never depend on script order. Listen for the event, and also
  // re-check on a timer, then give up rather than polling forever.
  if (consented()) { send(); return; }
  addEventListener("consentchange", function () { if (consented()) send(); });
  timer = setInterval(function () {
    if (consented()) send();
    if (++ticks > 60) { clearInterval(timer); timer = null; }
  }, 1000);
})();
</script>

And the receiving end:

// The /hit route. Append-only, cookieless, no PII.
// sendBeacon posts text/plain, so read the raw body rather than a JSON parser.
app.post("/hit", async (req, res) => {
  res.status(204).end();               // answer first, never make the page wait
  try {
    const d = JSON.parse(await raw(req));
    if (!d || !d.s) return;
    appendFileSync("./hits.jsonl", JSON.stringify({
      t: new Date().toISOString(),
      s: String(d.s).slice(0, 24),
      p: String(d.p || "").slice(0, 300),
      r: String(d.r || "").slice(0, 120),
      us: d.us, um: d.um, uc: d.uc, ut: d.ut,
      src: classify(d.r, d.us),
    }) + "\n");
  } catch (e) {}
});

// Classify SERVER-side, so the rule can be corrected retroactively against
// history you already hold. Classifying in the browser bakes in the mistake.
const AI_HOSTS = [
  "chatgpt.com", "chat.openai.com", "perplexity.ai", "copilot.microsoft.com",
  "gemini.google.com", "claude.ai", "you.com", "phind.com", "poe.com",
];
function classify(ref, utmSource) {
  if (utmSource) return "campaign";
  if (!ref) return "direct";
  if (AI_HOSTS.some((h) => ref.endsWith(h))) return "ai";
  if (/google\.|bing\.|duckduckgo\.|yahoo\./.test(ref)) return "organic";
  if (/facebook\.|instagram\.|linkedin\.|t\.co|reddit\./.test(ref)) return "social";
  return "referral";
}

Fire on every visit. My first version only fired when the referrer was one of ten AI hostnames. That silently discarded every direct, organic, social and campaign visit before it was ever sent. The traffic report sat nearly empty for weeks and I assumed the sites had no traffic. There was nothing to recover, because the data was thrown away in the browser.

Classify server-side, for the same reason. A classification rule applied in the browser is baked into history and cannot be corrected. Store the raw referrer hostname and decide what it means at read time.

Stage 5: the multi-tenant dashboard

One page per brand, each behind its own token, so a client sees their numbers and nobody else's.

Build me a dashboard that reads the JSONL files and shows
one page per site.

- Sites come from brands.json. Access tokens live in access.json, one token per
  dashboard, chmod 600. An admin token can see every site; a client token sees
  exactly one. Never let one token read another site's numbers.
- Scope every total by site AT READ TIME, not when writing the data. Keep the
  raw values on each row so that if I get the scoping wrong I can fix it and the
  correction applies to all the history I already have.
- If another of my own sites was cited instead of this one, show that as its own
  state. Do not count it as a win for the site I am looking at.
- Show a dash, never a zero, for anything that was never captured. A gap and a
  measured zero mean opposite things: one means fix the collector, the other
  means fix the site.
- Put a STALE badge on any feed whose newest row is more than three days old.

Then show me the dashboard for my lowest-traffic site, not my busiest one. A
scoping bug is invisible on the site that legitimately has all the numbers.

You should see a dashboard you can open in a browser, showing only the site whose token you used. Try a second site's token and confirm the numbers change completely. If they do not, the scoping is wrong, and that is the single most important thing to catch here.

The two config files it should end up with:

// brands.json, the single source of truth for the site list.
// Every collector READS this. Nothing hardcodes domains, or they drift apart.
{
  "acme":  "acmeinsurance.com",
  "beta":  "betabenefits.com",
  "gamma": "gammahealth.org"
}

// access.json, one token per dashboard, chmod 600.
// An admin token lists every brand; a client token lists exactly one.
{
  "b7f2...": { "label": "admin",       "brands": ["acme", "beta", "gamma"] },
  "3a91...": { "label": "Acme client", "brands": ["acme"] }
}

Scope every aggregate by brand, at read time. This is the bug I am least proud of. A metric computed across the whole portfolio was rendered on a single brand's page, so a sister property being cited scored as a win on the wrong dashboard and dragged its sources along with it. Because the raw cited array was preserved on every row, the fix corrected all history retroactively. Had I stored only the computed boolean, months of data would have been wrong permanently.

// Scope EVERY aggregate by brand at read time, not at write time.
// Keep the raw cited[] array and a scoping bug stays correctable forever.
function brandData(brandId, rows) {
  const domain = BRANDS[brandId];
  const sisters = Object.entries(BRANDS)
    .filter(([id]) => id !== brandId)
    .map(([, d]) => d);

  return rows.map((row) => {
    const citedUs     = (row.cited || []).some((u) => host(u) === domain);
    const citedSister = (row.cited || []).some((u) => sisters.includes(host(u)));
    return {
      ...row,
      citedUs,
      // A sister property being cited is NOT a win for this brand. Render it
      // as its own state instead of letting it inflate the count.
      sisterOnly: !citedUs && citedSister,
    };
  });
}

Two display rules worth insisting on:

• Show a dash, never a zero, for a metric that was never captured. A gap and a measured zero mean opposite things. One means go and fix the collector, the other means go and fix the site. A zero in both places guarantees you eventually act on the wrong one.
• Put a stale badge on any feed older than three days. Frozen data that still looks current is the failure mode that hides every other failure mode, and stage 6 is the feed most likely to freeze.

After you build it, test with a low-traffic brand rather than your busiest one. A scoping bug is invisible on the brand that legitimately has all the numbers.

Stage 6: automate the AI capture

There is no API for the AI report, so this stage drives a real browser holding its own signed-in profile, on a schedule. It is the fiddliest part of the build and it will need attention forever, because sessions expire.

It cannot run on your server. The AI endpoints authenticate off HttpOnly Microsoft session cookies, which cannot be exported to a machine that is not the one that signed in. So this runs on a workstation, headless, on a scheduled task.

Automate the Bing AI Performance capture so I do not have to
paste anything into a browser console every day.

It has to drive a real browser, because those endpoints authenticate off
Microsoft session cookies that are HttpOnly and cannot be copied to a server.

- Use Playwright with a persistent Chromium profile stored in ./profile, so the
  sign-in survives between runs.
- Give me a --login mode that opens a visible window so I can sign in once, and
  a normal headless mode for the daily run.
- Do the fetches INSIDE the page, where the cookies live.
- Important: HTTP 401 means nobody is signed in. HTTP 403 means somebody IS
  signed in but that account does not own these sites. Treat those as DIFFERENT
  outcomes with different messages. If you collapse them, the tool tells me to
  sign in while I am already signed in and I will loop on it forever.
- Exit codes: 0 success, 2 signed out, 3 failed, 4 wrong account. My scheduler
  reads the exit code, and if a failure exits 0 it runs green while capturing
  nothing.
- Write the last outcome to a small JSON status file every run, so I can see
  when it last actually worked without reading a log.

Then show me how to schedule it to run daily on my computer.

You should see a browser window open once for sign-in, then close by itself. After that, the headless run should print the same citation totals you saw by hand in stage 1. If it exits 2 it is signed out. If it exits 4 you are signed into the wrong Microsoft account, which is a different problem with a different fix, and telling them apart is the whole point of the next paragraph.

What it should produce:

// capture.mjs, the daily automated version of stage 1.
//
// WHY A BROWSER: the AI endpoints authenticate off HttpOnly Microsoft session
// cookies, which cannot be exported to a server. So the job drives a real
// Chromium that keeps its own signed-in profile on disk.
//
//   node capture.mjs --login     one-time, visible browser, sign in
//   node capture.mjs             headless capture and ingest
//   node capture.mjs --dry-run   capture and print, post nothing
//
// Exit codes: 0 ok · 2 signed out · 3 failed · 4 wrong account
import { chromium } from "playwright";

const ctx = await chromium.launchPersistentContext("./profile", {
  headless: !LOGIN,
  viewport: { width: 1440, height: 900 },
  args: ["--disable-blink-features=AutomationControlled"],
});
const page = ctx.pages()[0] || (await ctx.newPage());
await page.goto("https://www.bing.com/webmasters/aiperformance", {
  waitUntil: "domcontentloaded", timeout: 90000,
});

// Everything runs INSIDE the page, where the session cookies live.
const res = await page.evaluate(async (SITES) => {
  const tok = (await (await fetch("/webmasters/auth/token",
    { credentials: "include" })).text()).trim();
  if (!/^[a-f0-9]{16,}$/i.test(tok)) return { signedOut: true };

  const H = { "Content-Type": "application/json", "X-CSRF-Token": tok };
  const post = async (p, body) => {
    const r = await fetch(p, {
      method: "POST", headers: H, credentials: "include",
      body: JSON.stringify(body),
    });
    return [r.status, await r.text()];
  };

  const out = {};
  let attempted = 0, unauth = 0, forbidden = 0;

  for (const [id, dom] of Object.entries(SITES)) {
    const siteUrl = "https://" + dom + "/";
    attempted++;
    const [s1, b1] = await post("/webmasters/api/aiperformance/citationstats", { siteUrl });

    // 401 means NOBODY is signed in.
    // 403 means someone IS signed in, but that account does not own this site.
    // Collapsing the two is why "sign in again" can loop forever.
    if (s1 === 401 || s1 === 403) {
      if (s1 === 401) unauth++; else forbidden++;
      continue;
    }
    if (s1 !== 200) continue;

    const stats = JSON.parse(b1).CitationStats || [];
    const series = stats.map((x) => ({
      d: x.Date.slice(0, 10), v: x.Citations, pages: x.UniqueCitedPages,
    }));

    const [s2, b2] = await post("/webmasters/api/aiperformance/searchqueries/stats", { siteUrl });
    const queries = s2 === 200 ? JSON.parse(b2).Queries || [] : [];

    out[id] = {
      totalCitations: series.reduce((a, x) => a + x.v, 0),
      window: series.length
        ? series[0].d + " to " + series[series.length - 1].d
        : "no data",
      series: series.map(({ d, v }) => ({ d, v })),
      groundingQueries: queries
        .map((q) => ({
          q: q.GroundingQuery,
          intent: (q.Intent || []).join(", "),
          citations: q.Citations,
          share: Math.round(q.CitationRate * 1000) / 10 + "%",
        }))
        .sort((a, b) => b.citations - a.citations),
      capturedAt: new Date().toISOString(),
    };
  }

  if (attempted && unauth + forbidden === attempted) {
    return { signedOut: true, wrongAccount: forbidden > 0 && unauth === 0 };
  }
  return { payload: out };
}, SITES);

Distinguish 401 from 403. This one cost me the most. HTTP 401 means nobody is signed in. HTTP 403 means somebody is signed in, but that account does not own these sites. Collapsing both into "signed out" produces a tool that tells you to sign in again while you are already signed in, which is an unbounded loop. If you have a personal and a work Microsoft account, this will happen to you.

Make signed-out its own exit code so your scheduler surfaces it rather than swallowing it. Mine ran green for two days while capturing nothing, because a failed capture and a successful one both exited 0. Use 0 for success, 2 for signed out, 3 for a real failure and 4 for wrong account, then check the task's last result:

Get-ScheduledTaskInfo -TaskName 'AI Capture'   # LastTaskResult, NextRunTime
Start-ScheduledTask   -TaskName 'AI Capture'   # run it now

Also write the last outcome to a small JSON file on every run. When this breaks, you want to know when it last worked without reading a log.

Stage 7: cross-channel reconciliation

This is the stage that makes the other six worth building, and I built it last, which was a mistake. Every channel was already being collected and none of them were ever compared. A property can sit for months with heavy AI citation and almost no presence on Google, and nothing anywhere trips, because no single dashboard is looking at two numbers at the same time.

The rule is simple: strong signal in one channel and silence in another is an error, not a curiosity.

Last piece, and it is the one that makes the rest worth having.

Write me a script that reads every feed I now collect and flags where they
CONTRADICT each other. Right now each console looks fine on its own, which is
exactly how a site sits broken for months.

Raise a flag when:
- the sitemap I actually serve and the sitemap Search Console thinks it has are
  more than 20% apart
- a site has real Bing traffic or AI citations but almost nothing from Google
- a site has pages in the Bing index but zero impressions, which usually means
  unknown URLs return 200 with the homepage instead of a 404
- a site has lots of impressions and zero clicks
- a site has no beacon hits at all in 30 days, which means every other number
  for it is unverified
- a site is at zero on every channel

For each flag give me a short plain-English line saying what it probably is and
what to check first. Then print a summary: how many sites are clean, how many
need attention.

Run it across all my sites and walk me through the results.

You should see a list of your sites with flags against them, and it is normal for this to be uncomfortable the first time. Sites you assumed were fine will come back flagged, because until now nothing was comparing the consoles to each other.

The logic it should produce:

// reconcile.mjs, the layer that makes the other six worth building.
// Every channel was already collected. None of them were ever COMPARED.
export function reconcile(brand, f) {
  const flags = [];

  // The sitemap you serve versus the sitemap Google thinks it has.
  if (f.sitemapUrls && f.gscSubmitted &&
      Math.abs(f.sitemapUrls - f.gscSubmitted) / f.sitemapUrls > 0.2) {
    flags.push({ code: "SITEMAP_DRIFT", level: "critical",
      detail: f.sitemapUrls + " served vs " + f.gscSubmitted + " submitted" });
  }

  // Strong on one engine, invisible on the other. Two different failures that
  // look identical if you only ever open one console.
  if ((f.bingImpressions > 50 || f.aiCitations > 50) && f.googleImpressions < 50) {
    flags.push({ code: "GOOGLE_BLIND", level: "critical",
      detail: "cited by AI and ranking in Bing, effectively absent from Google" });
  }
  if (f.bingInIndex > 20 && f.bingImpressions === 0) {
    flags.push({ code: "BING_BLIND", level: "critical",
      detail: f.bingInIndex + " pages indexed, zero impressions (the soft-404 signature)" });
  }

  if (f.googleImpressions > 500 && f.googleClicks === 0) {
    flags.push({ code: "NO_CLICKS", level: "warn",
      detail: "ranking but never chosen: titles, metas or cannibalisation" });
  }

  // Without the beacon, every other number here is unverified.
  if (f.beaconHits30d === 0) {
    flags.push({ code: "NO_BEACON", level: "warn",
      detail: "snippet missing or gated shut" });
  }

  if (!f.inGsc)            flags.push({ code: "NOT_IN_GSC", level: "critical" });
  if (!f.sitemapReachable) flags.push({ code: "SITEMAP_UNREACHABLE", level: "critical" });

  if (!f.googleImpressions && !f.bingImpressions && !f.aiCitations && !f.beaconHits30d) {
    flags.push({ code: "DARK", level: "critical", detail: "zero signal on every channel" });
  }

  return { brand, flags, clean: flags.length === 0 };
}

Those thresholds are not arbitrary. Each corresponds to a distinct, diagnosable failure:

Run this across a portfolio and it surfaces things no single console will show you. On its first pass over ours it raised six critical flags, two of them exact opposite failures: one set of pages strong in Bing and invisible in Google, another set healthy in Google and completely dead in Bing. Read one console at a time and both present as "traffic is a bit soft". That is the reconciler earning its place.

The full cron schedule

Order matters. The reconciler has to run after the collectors it reads, or it compares today's numbers against yesterday's and flags things that are not real.

# Collectors run in dependency order, then the reconciler compares them.
10 13 * * *  cd /opt/collector && node panel/prompt-panel.mjs    >> logs/panel.log 2>&1
20 13 * * *  cd /opt/collector && node consoles/bing-pull.mjs    >> logs/bing.log  2>&1
30 13 * * *  cd /opt/collector && node consoles/gsc-pull.mjs     >> logs/gsc.log   2>&1
40 13 * * *  cd /opt/collector && node consoles/history-pull.mjs >> logs/hist.log  2>&1
45 13 * * *  cd /opt/collector && node collector/recon-pull.mjs  >> logs/recon.log 2>&1

# Stage 6 runs on a workstation, not here: it needs the signed-in browser
# profile. Windows Task Scheduler, daily, "start when available".
#   schtasks /Create /TN "AI Capture" /TR "C:\path\run.cmd" /SC DAILY /ST 07:15

Seven traps, and how to clear them

The fiddly part of this build is not any one collector, it is merging four APIs that disagree about dates, identity and authentication into one consistent history. These are the seven that take the longest to diagnose. Each is cheap to avoid once you know it is coming. Read them before you start stage 3.

1. Cross-brand aggregates. A metric computed across every site but rendered on one site's page will credit one property with citations that belong to another. Scope every aggregate by brand in the query itself, then open your quietest property and confirm it has not inherited a busy one's numbers.
2. Filtering at the browser. A beacon that only fires on AI referrals discards almost every real visit before you can classify it, and you cannot recover what was never sent. Fire on every visit, send the raw referrer, and decide what it means server-side, where you can change your mind later.
3. Auth tokens that lie. A signed-out session still hands you a valid-looking token, so a token check reports healthy while the collector quietly returns nothing. Make the health check a real data call and assert on the payload.
4. 403 means the wrong account, not a signed-out one. Read literally it sends you round the sign-in loop indefinitely. The fix is to sign out first, then back in with the account that owns the property.
5. A site list held in more than one place. Three collectors each with their own copy means adding a site updates two of them and leaves the third quietly wrong for weeks. One brands.json, read by everything.
6. Regex that accepts either quote character. A pattern matching ["'] stops at the first apostrophe inside the value, so Here's why truncates to Here and a set of perfectly good meta descriptions looks broken. Capture the opening quote and match that same character.
7. Rounded values used as identity. Group records by a rounded figure and records that genuinely differ collapse together, so you report a match that is not there. Group by the whole record and round for display only.

Questions