Who operates this crawler
Webb9Bot is operated by Webb9 (webb9.com). It identifies itself honestly on every request — no browser impersonation, no rotating user agents, no residential proxies:
User-Agent: Webb9Bot/1.0 (+https://webb9.com/bot)
From: bot@webb9.comWhat it collects
Only what a business already publishes on its own public web pages: business name, services, phone number, address, opening hours, published color scheme, public social links, public descriptions, service locations, logo URL, and page titles/metadata.
Never collected: anything behind a login or a form, personal data beyond public business contact details, or content from pages marked noindex/nofollow. Named-individual email addresses (e.g. a person's first-name address) are never collected — only role addresses already published for the business itself (such as info@ or contact@), and only after independent verification.
Why
Webb9 builds a website redesign concept from a business's own public information and sends it to that business as outreach — a demonstration of what a modernized site for their business could look like, not their official website. Every such concept is clearly labeled as a Webb9 concept, not affiliated with or operated by the business.
How this crawler behaves
robots.txt is fetched before any other request on a host and honoured unconditionally, including Disallow and Crawl-delay. Beyond that, the crawler limits itself to at most 5 pages per business, a 30-second total budget, one request in flight per host at a time with at least 2 seconds between requests, a 10-second timeout per request, and a 2MB cap per page.
How to block it
- Add a
Disallowrule forWebb9Bot(or*) to your site'srobots.txt— honoured on every future visit, automatically. - Email bot@webb9.com with your business name and website. This permanently blocks future crawling of your site and removes any existing concept built from it.
How to request deletion
Email bot@webb9.com and identify your business. On request, Webb9 deletes everything derived from crawling your site: extracted facts, generated content, screenshots, and any access tokens for a concept built from it. A record that the request was made and honoured is kept, the same way a purge is recorded for any other retention policy on this platform.
Requests are reviewed and actioned by Webb9 staff rather than processed instantly and automatically — email is the reliable channel today.
Contact
Questions about this crawler: bot@webb9.com. General privacy questions: privacy@webb9.com.