GovHubBot

GovHubBot is the crawler behind GovHub's bid inbox. It reads the public bid and solicitation listings that government agencies post on their own websites, so small businesses can find opportunities that fit them, and it links each listing back to the agency's page.

How to recognize it

Every request carries this user agent:

Mozilla/5.0 (compatible; GovHubBot/1.0; +https://www.govhub.online/bot/)

Its direct requests are also signed with Web Bot Auth (HTTP Message Signatures). Each one carries Signature-Agent, Signature-Input and Signature headers, and the public key that verifies them is published at https://www.govhub.online/.well-known/http-message-signatures-directory. A request that claims to be GovHubBot without a valid signature is not ours.

What it reads

How it behaves

How to limit or stop it

To keep GovHubBot off your whole site, add this to your robots.txt:

User-agent: GovHubBot
Disallow: /

You can also disallow single paths, or set a Crawl-delay in seconds. Changes take effect on its next daily visit. If you would rather tell us directly, or want a listing corrected or removed, email [email protected] and we will act on it promptly.

Allowing it through bot protection

If your site sits behind bot protection that blocks GovHubBot and you would like your bids listed, you can allow requests whose signature verifies against the key directory above, or match on the GovHubBot user agent. Email [email protected] if you need anything else from us to do that.