We shipped the Cloudflare integration to track AI agents and bots at the edge, giving you raw insights that GA4 often misses.
Highlights
- Track AI crawlers (GPTBot, ClaudeBot, PerplexityBot, etc.) directly from Cloudflare logs.
- Monitor bot health scores and HTTP status code distribution.
- Verify bot authenticity with Cloudflare's verification system (AEO Trust Score).
- View real-time activity feed of AI bot crawls across your content.
- Analyze content distribution to see which pages AI agents crawl most.
What's included
- Bot Analytics: Total AI crawls, bot health scores, and primary AI agent identification.
- Verification Tracking: Distinguish verified bots from likely automated scrapers.
- Status Code Monitoring: Track 2xx, 3xx, 4xx, and 5xx responses by bot type.
- Error Analysis: Identify top error paths and error rate trends over time.
- Content Distribution: See which site sections receive the most AI bot traffic.
- Activity Feed: Real-time view of recent AI bot activity with timestamps and paths.
How teams use it
- Monitor AI crawler activity before traffic reaches your origin server.
- Identify bot authentication issues and firewall blocks affecting AI discovery.
- Track which content AI agents are actively crawling and indexing.
- Debug errors that prevent AI bots from accessing your content.
- Compare bot verification rates to ensure legitimate AI traffic.