```text # ========================================================== # robots.txt for https://www.syeds.com.pk/ # Optimized for WordPress, SEO, AI Search and AI Crawlers # ========================================================== # ---------------------------------------------------------- # Default Rules for Search Engines and Crawlers # ---------------------------------------------------------- User-agent: * Allow: / # Protect WordPress administration Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php # Prevent crawling of internal search result URLs Disallow: /?s= Disallow: /search/ Disallow: /*?s= Disallow: /*&s= # ---------------------------------------------------------- # OpenAI # ---------------------------------------------------------- User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / # ---------------------------------------------------------- # Anthropic / Claude # ---------------------------------------------------------- User-agent: ClaudeBot Allow: / User-agent: Claude-SearchBot Allow: / User-agent: Claude-User Allow: / User-agent: Anthropic-ai Allow: / # ---------------------------------------------------------- # Google Search and AI # ---------------------------------------------------------- User-agent: Googlebot Allow: / User-agent: Google-Extended Allow: / User-agent: GoogleOther Allow: / # ---------------------------------------------------------- # Microsoft / Bing # ---------------------------------------------------------- User-agent: Bingbot Allow: / # ---------------------------------------------------------- # Apple # ---------------------------------------------------------- User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / # ---------------------------------------------------------- # Perplexity AI # ---------------------------------------------------------- User-agent: PerplexityBot Allow: / # ---------------------------------------------------------- # Meta # ---------------------------------------------------------- User-agent: Meta-ExternalAgent Allow: / User-agent: Meta-ExternalFetcher Allow: / User-agent: FacebookBot Allow: / # ---------------------------------------------------------- # Amazon # ---------------------------------------------------------- User-agent: Amazonbot Allow: / # ---------------------------------------------------------- # Cohere # ---------------------------------------------------------- User-agent: cohere-ai Allow: / # ---------------------------------------------------------- # Mistral AI # ---------------------------------------------------------- User-agent: MistralAI-User Allow: / # ---------------------------------------------------------- # ByteDance # ---------------------------------------------------------- User-agent: Bytespider Allow: / # ---------------------------------------------------------- # Other Crawlers # ---------------------------------------------------------- User-agent: CCBot Allow: / User-agent: Diffbot Allow: / User-agent: PetalBot Allow: / User-agent: DuckAssistBot Allow: / User-agent: AI2Bot Allow: / User-agent: Brightbot Allow: / User-agent: Omgilibot Allow: / User-agent: PhindBot Allow: / User-agent: DataForSeoBot Allow: / # ---------------------------------------------------------- # XML Sitemap - Rank Math # ---------------------------------------------------------- Sitemap: https://www.syeds.com.pk/sitemap_index.xml ```