How to Find and Allow a Blocked Good Bot in Cloudflare

Why Use Cloudflare’s Web Application Firewall (WAF)?

A Web Application Firewall (WAF) is a crucial layer of security for any website, filtering out malicious traffic, bots, malware, and hack attempts before they even reach your server. Cloudflare’s WAF provides an effective shield against common threats like SQL injections, cross-site scripting (XSS), and brute force attacks. Best of all, it’s included in Cloudflare’s Free Plan, allowing users to create up to five custom rules to fine-tune their security settings. By leveraging the WAF, website owners can maintain performance and uptime while blocking harmful traffic automatically.

However, while the WAF is excellent at keeping out malicious bots and suspicious activity, it can sometimes block legitimate bots and services as well. This happens when a bot or service’s User Agent is either unknown to Cloudflare or not classified as a verified bot. Some third-party services, monitoring tools, or API connections may not be recognized as bots at all, leading to unexpected blocking. When these services are essential for your website’s functionality—such as uptime monitoring, content scrapers for indexing, or API connections—they need to be added to the WAF allowlist to ensure they can connect without issues. This guide will show you how to identify a blocked bot and allow it in Cloudflare’s WAF settings.

Step 1: Check Cloudflare Security Events

1. Log in to your Cloudflare dashboard.
2. Select the website where the bot is being blocked.
3. Navigate to Security > Events.
4. Use the search bar or filters to look for events where the Action is “Block” or “Managed Challenge.”
5. Identify the relevant request based on the time of the request, and the path or user agent triggering the block.

Step 2: Identify the Bot’s User Agent

1. Click on the event to view detailed request logs.
2. Look for the User Agent field. This identifies the bot that was blocked.
3. Copy the text portion of the User Agent string.

Step 3: Add the Bot to the WAF Allow Rule

1. Go to Security > WAF.
2. Select Custom Rules and if present, click on the existing rule that skips your other WAF rules, or create a new rule named “Good Bots Allow”.
3. Set the condition:

• Field: “User Agent”
• Operator: “contains”
• Value: Paste the copied User Agent string. It will match partial strings, so “UAgent” matches “MyCo UAgent” and “UAgent 2.0”, etc.

1. Set the action to Allow.
2. Save or update the rule.

Now, the bot should no longer be blocked by Cloudflare’s security features. Always ensure that the bot is trustworthy before allowing it through your firewall!

Automate Bulk Sites Rule Creation with our Cloudflare WAF Wizard WordPress plugin

Instead of manually creating WAF rules and/or adding bots to your allowlist, you can automate this process using our Cloudflare WAF Wizard WordPress plugin. This tool connects via Cloudflare’s API and creates all the necessary rules in bulk across multiple domains at once. Simply select or paste in the User Agents you want to allow, and they’ll be automatically excluded from all other WAF rules, saving you time and effort. More details with screenshots at:

Download Cloudflare WAF Wizard