OPEN_SOURCE ↗
HN · HACKER_NEWS// 4h agoTUTORIAL
SurfacedBy maps AI traffic, referral clicks
Ali Khallad’s probe shows nginx logs can separate provider-side fetches from human clickthroughs, and that distinction matters if you’re measuring AI-driven traffic. The post also shows why Gemini, Copilot, and Grok are much harder to attribute cleanly than ChatGPT, Claude, or Perplexity.
// ANALYSIS
This is less a vanity traffic post than an observability warning: if you lump all “AI traffic” together, you’ll miss the real differences between bots that fetch your site, bots that answer from index, and humans who click back from AI citations.
- –ChatGPT, Claude, Perplexity, and Manus expose clearer retrieval signals; Gemini often does not, and Copilot/Grok can look like normal browser traffic.
- –A single `User-Agent` rule is not enough for bot detection or blocking because some vendors preflight `robots.txt`, fan out across multiple IPs, or hide behind ordinary browser headers.
- –The useful metric is not just provider fetches but referral visits from people clicking AI citations, which is the traffic most sites actually care about.
- –SurfacedBy is using the post to reinforce its GEO/AI visibility platform, so the article works both as technical research and as product marketing.
// TAGS
surfacedbynginxsearchllmchatbotinfrastructure
DISCOVERED
4h ago
2026-04-21
PUBLISHED
21h ago
2026-04-20
RELEVANCE
7/ 10
AUTHOR
startages