platforms
Shopify Robots.txt: Defaults, robots.txt.liquid and Safe Customization
When should a Shopify store edit robots.txt.liquid, and which default rules should be preserved?
Read guide →Knowledge base
Platform Robots.txt Guides
Platform-specific robots.txt guidance for Shopify, Next.js, Wix, Squarespace, Webflow, and Magento / Adobe Commerce, including defaults, editors, metadata routes, deployment behavior, and indexing controls.
Platform Robots.txt Guides guides
platforms
Next.js Robots.txt: How we Configure App Router Crawling Safely
Should we use app/robots.txt or app/robots.ts in Next.js, and how do we keep previews from blocking production?
Read guide →
platforms
Wix Robots.txt: How we Edit Rules Without Breaking Search Visibility
Where do we edit robots.txt in Wix, and when should we use page indexing settings instead?
Read guide →
platforms
Squarespace Robots.txt: How we Work With Platform-Managed Crawl Rules
What can we actually change about robots.txt in Squarespace, and which controls should we use instead?
Read guide →
platforms
Webflow Robots.txt: How we Configure Rules, Sitemap and AI Signals
How do we edit and publish Webflow robots.txt without duplicating sitemap rules or confusing AI content signals?
Read guide →
platforms
Magento Robots.txt: How we Audit Adobe Commerce Crawl Settings Safely
How should we configure Magento or Adobe Commerce robots.txt without copying a generic ecommerce template?
Read guide →