Robots (strong)

Download

About

Stronger plugin against robots for Bludit CMS.

Adds metatags to site head of all pages, including home page and category lists. And also adds robots.txt that blocks access from bots.

THIS PLUGIN CAN BLOCK NOT ALL CRAWLERS OR ROBOTS ACCESS, INDEXING, OR DATA SCRAPING. USE THIS PLUGIN AT OWN LISK.

Modes

Strong mode (default)

Avoid search engine indexing.

Adds these metatags to site head

<meta name="GOOGLEBOT" content="NOINDEX, NOFOLLOW, NOARCHIVE, NOIMAGEINDEX">
<meta name="ROBOTS" content="NOARCHIVE, NOINDEX, NOFOLLOW, NOIMAGEINDEX">
<meta name="robots" content="noindex, nofollow, noarchive, noimageindex">

Adds these content to robots.txt.

User-agent: *
Disallow: /

Strongest mode

Hard protection against AI crawler. In strongest mode, uses explicitly defined robots.txt rules and service-specific metatags.

Metatags to add:

<meta name="GOOGLEBOT" content="NOINDEX, NOFOLLOW, NOARCHIVE, NOIMAGEINDEX">
<meta name="ROBOTS" content="NOARCHIVE, NOINDEX, NOFOLLOW, NOIMAGEINDEX">
<meta name="robots" content="noindex, nofollow, noarchive, noimageindex, noimageai, noai">
<meta name="bingbot" content="noarchive">
<meta name="msnbot" content="noarchive">
<meta name="pinterest" content="nopin">

Adds robots.txt rules that disallows access to:

• GPTBot
• ChatGPT-User
• Google-Extended
• GoogleOther
• GoogleOther-Image
• GoogleOther-Video
• CCBot
• ClaudeBot
• PerplexityBot
• Applebot-Extended
• OAI-SearchBot
• amazon-kendra
• ImagesiftBot
• Diffbot
• meta-externalagent
• meta-externalfetcher
• cohere-ai
• AI2Bot
• Bytespider
• ICC-Crawler

License

MIT

Credit

• Robots
  Bludit default Robots plugin
• 個人サイトをAIに学習されるのを拒否しよう | 古の夢の住人 (JA)
  Metatags and robots.txt source