Our crawler

FirmLeadBot

FirmLeadBot/1.0 is the web fetcher of FirmLead. It reads a company's public website once, at a user's request, to summarise what the company does. It fetches a few pages at most, keeps only visible text, and never submits forms or logs in. If a site's bot protection refuses the identified request, it retries once with a standard browser user agent, still within the robots.txt rules below.

It identifies itself as:

Mozilla/5.0 (compatible; FirmLeadBot/1.0; +https://firmlead.io/bot)

To Block It

User-agent: FirmLeadBot
Disallow: /

Questions: info@firmlead.io