robotstxt-ai
io.github.sharozdawa/robotstxt-ai
About robotstxt-ai
robotstxt-ai is an MCP server published by the GitHub account sharozdawa. Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.
It runs on your own machine: your AI client starts it from the npm package robotstxt-ai-mcp at version 1.0.1.
It asks for no keys or passwords.
Version 1.0.1 was published on 21 March 2026, and the source code is public, so you can read it before installing.
Written from the registry entry. 9 automated checks on the registry entry and the package code: 7 pass, 0 warn, 0 fail, 2 info. How we check.
What it asks for
This server declares no environment variables, so it asks for no keys or settings.
Checks
| Check | Result | Detail |
|---|---|---|
| Source repository linked | Pass | https://github.com/sharozdawa/robotstxt-ai |
| Version pinned | Pass | Every package has an explicit version |
| Secrets requested | Info | Asks for no secrets |
| Known package registry | Pass | All packages come from npm, PyPI, OCI, NuGet or MCPB |
| Namespace ownership | Info | Name verified via GitHub account sharozdawa |
| Registry text aimed at the AI | Pass | Description and settings contain no hidden characters or instructions to the AI |
| Install scripts | Pass | robotstxt-ai-mcp@1.0.1 runs nothing at install |
| Code red flags | Pass | Read 6 files in robotstxt-ai-mcp@1.0.1: no obfuscation, encoded execution, credential reads or data-collection endpoints found |
| Text aimed at the AI | Pass | No hidden characters or instructions to the AI in 6 files, including comments and the README |
Popularity & health: 39 / 100
Worked out from public numbers and our own checks, not from user reviews or ratings. It says how used and how looked-after a server is, not whether it is safe.
0 forks, 0 merged pull requests. Sources: GitHub, npm, PyPI (pypistats.org) and Hacker News (Algolia), fetched 2026-10-31.
What people say
No Hacker News threads link to its source code.
Look further: search Reddit, search Hacker News, read its GitHub issues. These are not reviews we checked.
Facts
| Registry name | io.github.sharozdawa/robotstxt-ai |
|---|---|
| Publisher | GitHub account sharozdawa |
| How the name is verified | Name verified via GitHub account sharozdawa. The registry ties the name prefix to that account or domain; it does not check the code. |
| Type | Local: runs on your machine from a package |
| Packages | npm robotstxt-ai-mcp 1.0.1 (stdio) |
| Version | 1.0.1 |
| Published | |
| Last updated | |
| Source repository | https://github.com/sharozdawa/robotstxt-ai |
| Website | None linked |
| Category | Browser automation (assigned by keyword rule, not by hand) |
Related servers in Browser automation
- Unrendered: AI Crawler Checker7/7AI crawler checker: shows which page content GPTBot and ClaudeBot…
- Crawlerpeek7/7Check whether AI crawlers (GPTBot, ClaudeBot, PerplexityBot) can…
- polite-crawler7/7A polite crawler for agents: obeys robots.txt, fetches each page…
- QueryRev AI Readiness7/7Checks a site for AI crawler access, robots.txt and llms.txt. Runs…
- CitedBy4/4Check a page, AI-crawler access, llms.txt and backlinks of a site.…
- agentbuilt4/4Free AI-readiness audit of any URL: AI crawler rules, JS-free text,…