Build a valid robots.txt with sitemap and disallow paths.
User-agent: * Allow: / Disallow: /admin Disallow: /checkout Disallow: /api/ Sitemap: https://example.com/sitemap.xml