A list of common crawler agents used on Internet..
Cross-referenced across 55 tracked directories
#2981
Popularity Rank
1 / 55
Listed In
Emerging
Adoption Stage
8/6/2023
Created
29
GitHub Stars
2
Forks
1,060
Weekly Downloads
10/5/2026
Last Commit
Recently added to the ecosystem
Score: 100/100
0 dependency vulnerabilities found
Headless browser automation server and OpenClaw plugin for AI agents - anti-detection, element refs, and session isolation
...moreGitHub Actions
An AI web browsing framework focused on simplicity and extensibility.
nstbrowser
Nstbrowser AI agent for browser automation with advanced fingerprinting
hello_sideguide
Official Firecrawl nodes for n8n - scrape, crawl, map, search, and extract data from websites. Supports AI Agent tool usage.
...more