Valida el archivo robots.txt y la caché de Google

• Caché en Google Search Console | robots.txt

Comprueba la fecha de la última indexación realizada por Google y solicita un nuevo rastreo del archivo robots.txt.

---

---

• Herramientas para consultar robots.txt en línea:

Robots.txt Tester: Check Your Robots.txt File for FreeSE Ranking

robots.txt Testing ToolTame The Bots

---

• Ejemplos estándar de robots.txt: https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt?hl=pt-br

User-agent: Applebot
Allow: /
Disallow: /cart
Disallow: /index.php?spsr=checkout
Disallow: /index.php?spsr=account
Disallow: /checkout
Disallow: /account

User-agent: Bingbot
Allow: /
Disallow: /cart
Disallow: /index.php?spsr=checkout
Disallow: /index.php?spsr=account
Disallow: /checkout
Disallow: /account

User-agent: Googlebot
Allow: /
Disallow: /cart
Disallow: /index.php?spsr=checkout
Disallow: /index.php?spsr=account
Disallow: /checkout
Disallow: /account

User-agent: Googlebot-Image
Allow: /
Disallow: /cart
Disallow: /index.php?spsr=checkout
Disallow: /index.php?spsr=account
Disallow: /checkout
Disallow: /account

User-agent: Mediapartners-Google
Allow: /
Disallow: /cart
Disallow: /index.php?spsr=checkout
Disallow: /index.php?spsr=account
Disallow: /checkout
Disallow: /account

User-agent: Adsbot-Google
Allow: /
Disallow: /cart
Disallow: /index.php?spsr=checkout
Disallow: /index.php?spsr=account
Disallow: /checkout
Disallow: /account

User-agent: AdsBot-Google-Mobile
Allow: /
Disallow: /cart
Disallow: /index.php?spsr=checkout
Disallow: /index.php?spsr=account
Disallow: /checkout
Disallow: /account

User-agent: Storebot-Google
Allow: /
Disallow: /cart
Disallow: /index.php?spsr=checkout
Disallow: /index.php?spsr=account
Disallow: /checkout
Disallow: /account

User-agent: Google-InspectionTool
Allow: /
Disallow: /cart
Disallow: /index.php?spsr=checkout
Disallow: /index.php?spsr=account
Disallow: /checkout
Disallow: /account

User-agent: GoogleOther-Image
Allow: /
Disallow: /cart
Disallow: /index.php?spsr=checkout
Disallow: /index.php?spsr=account
Disallow: /checkout
Disallow: /account

User-agent: msnbot
Allow: /
Disallow: /cart
Disallow: /index.php?spsr=checkout
Disallow: /index.php?spsr=account
Disallow: /checkout
Disallow: /account


User-agent: *
Disallow: /