# Voxarah — robots policy # We take our users' data seriously, so go away. # Full human-readable policy: https://www.voxarah.com/data-policy # Allow: legitimate search engine indexers and social-unfurl bots only. # Disallow: AI scrapers, model trainers, dataset crawlers, and everything else. # ---------- Search engines (allowed) ---------- # Only truly private surfaces are disallowed. Public marketing routes like # /parent, /advocate, /share, /learn (lesson previews) stay crawlable. User-agent: Googlebot Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: Googlebot-Image Allow: / User-agent: Googlebot-News Allow: / User-agent: Bingbot Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: Slurp Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: DuckDuckBot Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: Baiduspider Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: YandexBot Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: Applebot Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks # ---------- Social unfurl / link-preview bots (allowed) ---------- # These fetch a single URL to render Open Graph share cards. They do not # train models or build datasets. Blocking them kills FB/IG/Threads/X/ # LinkedIn/Slack/Discord previews when users share Voxarah links. User-agent: facebookexternalhit Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: Twitterbot Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: LinkedInBot Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: Slackbot Allow: / User-agent: Slackbot-LinkExpanding Allow: / User-agent: Discordbot Allow: / User-agent: WhatsApp Allow: / User-agent: TelegramBot Allow: / # ---------- AI answer-time search bots (allowed) ---------- # These fetch a URL at query time to cite it in an AI answer. They do NOT # train models. Blocking them removes Voxarah from ChatGPT Search, Claude # answers, Perplexity, SearchGPT, Google AI Overviews, and Apple Intelligence # citations. Same private-path disallows as Googlebot. User-agent: OAI-SearchBot Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: ChatGPT-User Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: SearchGPT Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: Claude-SearchBot Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: Claude-User Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: PerplexityBot Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: Perplexity-User Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: Google-Extended Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: Applebot-Extended Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: DuckAssistBot Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks User-agent: MistralAI-User Allow: / Disallow: /dashboard Disallow: /onboarding Disallow: /settings Disallow: /unsubscribe Disallow: /certificate Disallow: /admin Disallow: /api/ Disallow: /hooks # ---------- AI training-data crawlers (blocked) ---------- # Notice to operators of the user agents below: # We take our users' data seriously, so go away. # Do not crawl, scrape, index, train on, fine-tune from, or otherwise ingest # any content on this site for training or dataset construction. Continued # access after this notice constitutes unauthorized use. User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: anthropic-ai Disallow: / User-agent: GoogleOther Disallow: / User-agent: GoogleOther-Image Disallow: / User-agent: GoogleOther-Video Disallow: / User-agent: CCBot Disallow: / User-agent: cohere-ai Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: Meta-ExternalFetcher Disallow: / User-agent: FacebookBot Disallow: / User-agent: Bytespider Disallow: / User-agent: TikTokSpider Disallow: / User-agent: Amazonbot Disallow: / User-agent: Diffbot Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: img2dataset Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: PanguBot Disallow: / User-agent: Timpibot Disallow: / User-agent: YouBot Disallow: / User-agent: AI2Bot Disallow: / User-agent: AwarioBot Disallow: / User-agent: AwarioRssBot Disallow: / User-agent: AwarioSmartBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: AhrefsBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: BLEXBot Disallow: / User-agent: PetalBot Disallow: / User-agent: Kangaroo Bot Disallow: / User-agent: Webzio-Extended Disallow: / User-agent: ISSCyberRiskCrawler Disallow: / User-agent: Scrapy Disallow: / User-agent: magpie-crawler Disallow: / User-agent: NewsNow Disallow: / User-agent: news-please Disallow: / User-agent: Sidetrade indexer bot Disallow: / User-agent: VelenPublicWebCrawler Disallow: / User-agent: ZoominfoBot Disallow: / User-agent: SeekportBot Disallow: / User-agent: TurnitinBot Disallow: / User-agent: NetEstate NE Crawler Disallow: / User-agent: Mistral-AI Disallow: / User-agent: xAI Disallow: / User-agent: Grok Disallow: / # ---------- Default policy ---------- # Notice: We take our users' data seriously, so go away. # Unknown bots are denied by default. Only the user agents explicitly listed # above are permitted. If your bot is not listed, you do not have permission # to crawl, scrape, index, train on, or otherwise access this site. User-agent: * Disallow: / Sitemap: https://www.voxarah.com/sitemap.xml