robots.txt Checker for Shopware & Shopify:
Is your store letting the right crawlers in?
A single line in your robots.txt decides whether Google finds your products, whether filter pages eat up your crawl budget and whether ChatGPT, Claude, Gemini and Perplexity can recommend your store. Mistakes often only show up once revenue is missing.
The checker detects Shopware 5, Shopware 6 and Shopify automatically, reads your robots.txt in real time and tests the rules against real pages of your store.
Check your robots.txt now
Enter your store address. You immediately see which shop system was detected, which crawlers get in and which rules cause problems. On request, you receive the full analysis with a corrected robots.txt as a PDF by email.
What your robots.txt has to get right.
Eight areas, tailored to Shopware 5, Shopware 6 and Shopify. Each area shows a place where a wrong rule costs visibility.
Availability and syntax
Does your store answer correctly at /robots.txt, or does it return errors, redirects or invalid lines that search engines interpret differently than intended?
Google and Bing
Are Googlebot and Bingbot allowed to access your homepage, categories and products, or does a rule block more than intended?
AI crawlers
Which crawlers from ChatGPT, Claude, Gemini and Perplexity get in? Separated into bots for training and bots that name your store in answers.
Filters, sorting and search
Do filter, sorting and search URLs eat up your crawl budget? Checked with the parameters Shopware and Shopify actually generate.
Resources
Are CSS, JavaScript or images blocked that Google needs to render your pages correctly?
Sitemap
Is your sitemap declared and reachable, and does it contain URLs that your robots.txt blocks at the same time?
noindex conflicts
Are pages blocked that carry a noindex? Then Google never sees the noindex, and the page can stay in the index anyway.
Real pages tested
Homepage, categories and products from your sitemap are tested against every rule. You see which pages are affected and by which line.
Where the robots.txt in Shopware and Shopify usually goes wrong.
Every shop system generates its own URLs and has its own place where the robots.txt is maintained. That is exactly where the typical mistakes happen.
Shopware 5
Many Shopware 5 stores carry rules from before AI crawlers existed that were never updated. The robots.txt is generated via a template in the theme. If you are planning the move to Shopware 6 or Shopify, set it up fresh right away.
Shopware 6
Depending on version and setup, Shopware 6 ships no robots.txt or only a generic one. With several sales channels and domains, every domain needs suitable rules, otherwise filter pages or wrong sitemaps end up being crawled.
Shopify
Shopify generates the robots.txt automatically and already blocks a lot sensibly. Changes are made via the robots.txt.liquid template in the theme. Most common mistake: a customised template overwrites the default rules, or filter and tag URLs of collections stay open.
Blocking everything costs visibility.
Not every AI crawler has the same job. Some collect content for training language models, others fetch your store when a customer asks ChatGPT, Claude, Gemini or Perplexity about a product. Block all AI bots and you disappear from those answers too. Allow all of them and you release more than necessary.
One example: if you block Google-Extended, Google no longer uses your content for Gemini. Your ranking in Google Search stays unaffected. The checker shows for every bot whether it gets in and what that means for your store. Whether AI assistants can reach your store technically is checked by the AI Bot Checker.
Get a quick read or a thorough review.
| Freerobots.txt Checker | robots.txt review by arboro | |
|---|---|---|
| Who checks | Automated check | Experts from the SEO/GEO team |
| Duration | A few seconds | Several hours of expert work |
| Data basis | robots.txt, sitemap and public pages | Additionally server logs and Google Search Console |
| Result | Score, key findings, PDF with corrected robots.txt | Reviewed configuration for all domains and sales channels |
| Implementation | By you or your agency | Directly by the arboro team |
robots.txt review by arboro
- Who checks
- Experts from the SEO/GEO team
- Duration
- Several hours of expert work
- Data basis
- Additionally server logs and Google Search Console
- Result
- Reviewed configuration for all domains and sales channels
- Implementation
- Directly by the arboro team
- Free
robots.txt Checker
- Who checks
- Automated check
- Duration
- A few seconds
- Data basis
- robots.txt, sitemap and public pages
- Result
- Score, key findings, PDF with corrected robots.txt
- Implementation
- By you or your agency
Have your robots.txt reviewed.
You want to know whether your robots.txt accidentally locks out search crawlers or allows AI crawlers more than Google? The arboro team reviews the configuration of your Shopware or Shopify store, cross-checks it with server logs and Search Console and implements the corrections directly.
 "Emmanuel Zouboulis")
Emmanuel Zouboulis
Head of SEO/GEO | Authorised Signatory
)
René Härer
Head of Content
 "Daniel Böttcher")
Daniel Böttcher
SEO/GEO Manager
)
Sebastian Lörz
Technical Online Marketing
 "Andra Schmidt")
Andra Schmidt
Senior Content Manager
)
Benjamin Breuer
Senior Content Manager
The people behind your analysis.
Behind the checker is the arboro SEO and GEO team. The full analysis and every further review are the responsibility of experts who optimise Shopware and Shopify stores for Google and AI systems, from the robots.txt to the code.
Frequently asked questions about the robots.txt Checker.
The robots.txt is a text file in the root directory of your store that tells crawlers which areas they may access. Search engines and most AI crawlers read it before visiting. It controls crawling, not indexing: a blocked page can still appear on Google if other pages link to it.
It checks whether your robots.txt guides search engines and AI crawlers through your store the way you want. It detects your shop system, evaluates every rule and tests it against real pages from your sitemap. Whether AI assistants can reach your store technically is shown by the AI Bot Checker. How well your store performs on Google is checked by the SEO Commerce Scanner.
Because generic testers do not know the specifics of these systems:
Shopware 5: robots.txt via a template in the theme, often with rules from before AI crawlers
Shopware 6: its own filter parameters, several sales channels and domains with their own rules
Shopify: automatically generated robots.txt, changes via robots.txt.liquid
arboro works exclusively with these systems. That is why the findings and the corrected robots.txt in the report are tailored to them.
Yes, the check and the short result are free and require no sign-up. For the full analysis as a PDF with a corrected robots.txt, all you need is your email address.
That depends on whether you want to be named in answers from ChatGPT, Claude, Gemini and Perplexity. Training bots can usually be blocked without your store disappearing from those answers. If you also block the bots that fetch content for answers, your store will no longer be recommended there. The report shows the consequence for every bot.
The checker only reads publicly accessible files and pages of your store, just as search engines do. Your email address is only used to send the analysis. You only receive the newsletter if you explicitly select it.
The full analysis contains a corrected robots.txt for your shop system and shows where it belongs. Your shop agency can implement much of it directly. If you would like support, the arboro team takes over review and implementation as part of SEO for online stores, even if your shop agency stays.
Your result is in. What now?
A wrong line is quickly fixed once you know which one it is. In the initial call, you discuss with an expert which findings cost your store the most and how to fix them.
 "Zoran Artmagic")
 "Shopware Gold Partner"){kind=logo}
 "Shopify Partner"){kind=logo}
 "Google Premier Partner"){kind=logo}
 "iBusiness Top 100 SEO"){kind=logo}
 "Internet Agency Ranking 2026")
 "Top Agency Performance Marketing 2026")
 "Owner-Managed Digital Agency 2026")
 "ISO 9001"){kind=logo}
 "TÜV Rheinland – Certified Qualification in Sales Psychology")