Crawlerdex
A field guide to the AI bots crawling the open web
Every crawler and user agent has its own behavior, purpose, and rules of engagement. Tap a card to inspect its Dex entry, base stats, and user-agent signature.
A field guide to the AI bots crawling the open web
Every crawler and user agent has its own behavior, purpose, and rules of engagement. Tap a card to inspect its Dex entry, base stats, and user-agent signature.