# rakidzich.com robots.txt # Public articles, assets, and discovery files are crawlable. # This file controls crawl requests; it is not an access-control mechanism. User-agent: * Allow: / Disallow: /*.bak Disallow: /pagefind/ Disallow: /*.tmp Allow: /search-index.json$ Disallow: /api Disallow: /cart Disallow: /seo Disallow: /search Disallow: /webhook Disallow: /articles_backup Disallow: /articles_articles_backup Disallow: /*utm_ Disallow: /*?offset= Disallow: /*&offset= Disallow: /*?s= Disallow: /*&s= Disallow: /*?q= Disallow: /*&q= Disallow: /*?format=rss Disallow: /*&format=rss # Applebot-Extended is more specific than Applebot and keeps the baseline policy. User-agent: Applebot-Extended Allow: / Disallow: /*.bak Disallow: /pagefind/ Disallow: /*.tmp Allow: /search-index.json$ Disallow: /api Disallow: /cart Disallow: /seo Disallow: /search Disallow: /webhook Disallow: /articles_backup Disallow: /articles_articles_backup Disallow: /*utm_ Disallow: /*?offset= Disallow: /*&offset= Disallow: /*?s= Disallow: /*&s= Disallow: /*?q= Disallow: /*&q= Disallow: /*?format=rss Disallow: /*&format=rss # Search and answer crawlers may use the public JSON search endpoint. User-agent: Googlebot User-agent: Applebot User-agent: DuckDuckBot User-agent: YandexBot User-agent: Bingbot User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: Claude-SearchBot User-agent: Claude-User User-agent: PerplexityBot User-agent: Perplexity-User Allow: /search Allow: / Disallow: /*.bak Disallow: /pagefind/ Disallow: /*.tmp Disallow: /api Disallow: /cart Disallow: /seo Disallow: /webhook Disallow: /articles_backup Disallow: /articles_articles_backup Disallow: /*utm_ Disallow: /*?offset= Disallow: /*&offset= Disallow: /*?s= Disallow: /*&s= Disallow: /*?q= Disallow: /*&q= Disallow: /*?format=rss Disallow: /*&format=rss Sitemap: https://www.rakidzich.com/sitemap.xml