# RUBIQ — robots.txt
# General crawl is welcome, and AI answer-engines are explicitly allowed (the GEO goal).
# Note: robots.txt is ADVISORY. Well-behaved crawlers honour it; bad bots ignore it, and some
# answer engines have been caught using undeclared crawlers. Real abuse control is edge bot
# management / WAF / rate-limiting (host-gated, Track B) — never put anything sensitive behind
# robots.txt alone. Sensitive utility paths (e.g. /api/, form confirmation pages) get a Disallow
# here once they exist (Track B), in addition to edge controls.

User-agent: *
Allow: /

# AI crawlers (allowed on purpose — RUBIQ wants to be discoverable in LLM answers)
User-agent: GPTBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: anthropic-ai
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /

Sitemap: https://rubiqbiz.com/sitemap-index.xml