# Hire4Higher Consulting - robots.txt # Default: allow everything. Block backend/internal paths and homepage design variants # that are kept around for reference (also noindex'd in the HTML ). User-agent: * Allow: / Disallow: /admin/ Disallow: /api/ Disallow: /cart/ Disallow: /home-v0 Disallow: /home-v1 Disallow: /home-v2 Disallow: /home-v3 Disallow: /home-v4 Disallow: /home-v5 # Modern AI crawlers (allow - we publish in llms.txt too) User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: PerplexityBot Allow: / Sitemap: https://hire4higherconsulting.com/sitemap-index.xml