BACK_TO_FEEDAICRIER_2
SurfacedBy maps AI traffic, referral clicks
OPEN_SOURCE ↗
HN · HACKER_NEWS// 4h agoTUTORIAL

SurfacedBy maps AI traffic, referral clicks

Ali Khallad’s probe shows nginx logs can separate provider-side fetches from human clickthroughs, and that distinction matters if you’re measuring AI-driven traffic. The post also shows why Gemini, Copilot, and Grok are much harder to attribute cleanly than ChatGPT, Claude, or Perplexity.

// ANALYSIS

This is less a vanity traffic post than an observability warning: if you lump all “AI traffic” together, you’ll miss the real differences between bots that fetch your site, bots that answer from index, and humans who click back from AI citations.

  • ChatGPT, Claude, Perplexity, and Manus expose clearer retrieval signals; Gemini often does not, and Copilot/Grok can look like normal browser traffic.
  • A single `User-Agent` rule is not enough for bot detection or blocking because some vendors preflight `robots.txt`, fan out across multiple IPs, or hide behind ordinary browser headers.
  • The useful metric is not just provider fetches but referral visits from people clicking AI citations, which is the traffic most sites actually care about.
  • SurfacedBy is using the post to reinforce its GEO/AI visibility platform, so the article works both as technical research and as product marketing.
// TAGS
surfacedbynginxsearchllmchatbotinfrastructure

DISCOVERED

4h ago

2026-04-21

PUBLISHED

21h ago

2026-04-20

RELEVANCE

7/ 10

AUTHOR

startages