Crawler

WebshopVitalsBot

Our crawler identifies itself as WebshopVitalsBot and links here. This page explains what it requests, how to allow it for a scan, and how to block it.

Last updated August 2026

01

Who we are

Webshop Vitals audits ecommerce stores for Merchant Center problems, product data inconsistencies, structured data issues, crawlability problems and AI shopping visibility opportunities.

Our crawler only visits a store when someone asks for a scan of it: either the owner from inside their account, or a visitor running a free scan of a URL they entered. We do not crawl the web speculatively.

02

How to identify us

Our requests carry the user agent WebshopVitalsBot/1.0 with a link back to this page.

We request pages over plain HTTPS GET. We read robots.txt first and respect a Disallow that applies to us.

We are rate limited per domain and run a small number of requests at a time, so a scan looks like a handful of ordinary page views rather than a flood.

03

How to allow WebshopVitalsBot

Allow this crawler identity in your firewall, CDN and WordPress security plugins: User agent: WebshopVitalsBot/1.0 Source IPs: • 208.77.244.241 • 152.55.184.241 • 152.55.185.189

If robots.txt blocks all crawlers, add this specific exception: User-agent: WebshopVitalsBot Disallow:

For SiteGround: open SiteGround Client Area → Help Center → Contact Us. Choose the closest technical-support category and ask them to allow the crawler for your domain. SiteGround runs its anti-bot system at server level, so changing robots.txt alone cannot clear a CAPTCHA block.

Copy this into the SiteGround request: Please allow WebshopVitalsBot/1.0 to access the public pages of my domain for an owner-requested Webshop Vitals audit. It uses HTTPS GET requests, respects robots.txt and does not submit forms. Please allow the user agent together with these source IPs: 208.77.244.241, 152.55.184.241, 152.55.185.189.

After SiteGround confirms the change, return to the failed scan and choose “I allowed it — scan again.” Firewall changes can take a few minutes to reach every server.

04

What we never do

We do not log in, submit forms, place orders, enter payment details or create accounts.

We do not attempt to bypass bot protection, solve CAPTCHAs or disguise ourselves as a browser to get around a block. If your firewall serves us a challenge page, we report the scan as blocked rather than scoring a page we could not read.

We do not follow links off your domain, and we do not store page HTML for longer than 7 days.

05

How to block us

Add the following to your robots.txt and we will stop on our next visit:

User-agent: WebshopVitalsBot Disallow: /

If you are seeing traffic you believe is not ours, or you want us to stop immediately, email support@webshopvitals.com with a sample log line and we will follow up.