Crawlability
Make sure Google can actually reach your product pages
A page that cannot be fetched cannot be ranked, sold or listed.
Everything else assumes Google can reach the page. Webshop Vitals checks that assumption first: robots rules, sitemap health including nested indexes, canonical logic, redirect chains and noindex tags, all evaluated against the product pages that carry your revenue.
Free scan, no credit card, results in minutes.
- High1 store
We could not find a return or refund policy page
No return or refund policy page was reachable from the homepage or at the conventional URLs we checked.
TRUST_RETURNS_MISSING - High11 products
Products are missing a product identifier
Branded items without a GTIN or MPN are limited by Google rather than disapproved, so the loss is invisible.
PD_IDENTIFIER_MISSING - Medium24 products
Shipping information is not machine readable
Delivery details appear in page text only, so they cannot be surfaced in results.
SD_SHIPPING_INFO_MISSING
Example findings, shown with the rule codes the product reports.and 2 more in the full report
The problem
The quietest failures happen before the crawl
A robots rule added for a staging site, a sitemap that stopped updating, a canonical that points every variant at a category page. None of these produce a visible error on the storefront, and all of them can remove products from search entirely.
- Product pages missing from search with no obvious reason
- A sitemap that has not updated since a migration
- Variants canonicalised into a single page
- Long redirect chains from an old URL structure
What we check
Inside the ecommerce crawlability audit
Every check below runs on your real store and reports findings you can act on, with the affected products named.
Robots rules per crawler
What each crawler is allowed to fetch, including rules that block a directory holding product assets.
Sitemap health
Discovery from robots.txt and conventional paths, nested sitemap indexes followed, and broken or unparseable sitemaps reported.
Index directives
Meta robots and header directives that quietly exclude pages from results.
Canonical logic
Whether canonicals point somewhere sensible, and whether variants are being collapsed incorrectly.
Redirect chains
Hops between the URL you publish and the page that finally answers.
Response health
Status codes, response times and pages that fail intermittently under a crawl.
Use this as your own checklist
0 of 5How it works
Three steps from question to answer
No plugin to install, no code to add, and nothing to change on your store.
- 1
Discover the real catalogue
Sitemaps declared in robots.txt are followed first, including sitemap index files, then internal links fill the gaps.
- 2
Fetch like a crawler
Requests respect rate limits and robots rules, and pages are rendered when they need it.
- 3
Report what blocks reach
Findings separate access problems from content problems so they can be fixed independently.
The report
What this check looks like in your dashboard
Findings land next to the score they moved and the products they affect, so a result always has somewhere to go next.
Website Readiness
89out of 100
- Product structured data30% 76
- Crawlability20% 97
- Product content20% 98
- Trust and merchant info15% 87
- AI commerce readiness15% 96
Products
Sorted by data quality| Product | Quality | Status |
|---|---|---|
3D & Collagen Snow Solution SKU ISOV-3DC-01 | 100 | Approved |
pH Balance Cleansing Cream SKU CORTHE-PH-08 | 80 | Limited |
Anti-Wrinkle Eye Mask SKU not set | 55 | Disapproved |
Holy Grow Lash Serum SKU BLOSSCO-LS-02 | 100 | Approved |
Since your last scan
- 7
- Issues fixed
- 2
- New issues
- 1
- Returned
- +4
- Readiness
Structured data returned on 24 products after a theme update. Caught the same day.
What you get
What changes after the first scan
Concrete outputs, not a grade with no next step.
- Certainty that your product pages can be fetched at all
- Sitemap problems found including nested index files that many tools skip
- Accidental noindex and blocked directories caught early
- A technical baseline before you invest in content or feed work
Questions
Ecommerce Crawlability Audit questions, answered
The things merchants ask before they run their first scan.
A sitemap index file lists other sitemaps rather than URLs, and a tool that does not follow the nested level sees an empty result. Compression and stylesheet declarations trip up other tools. Webshop Vitals follows nested indexes and handles compressed responses, and it names the exact file when something genuinely fails to parse.
Keep reading
Related checks
Product Structured Data Checker
Validate Product and Offer markup across every product page, not one URL at a time.
AI Commerce Readiness
Measure whether AI assistants can reach, read and understand your product data.
Automated Store Monitoring
Re-check on a schedule and get told what changed since the last scan.
See where your store actually stands
Run a free scan and get a prioritised report of what to fix first, with the affected products named.
No credit card needed for the free scan.