Azure Web Application Firewall- Microsoft_BotManagerRuleSet_1.0

rohith v 0 Reputation points
2023-11-08T07:31:22.3733333+00:00

This post is regarding the azure WAF unknown bots and its rules at the moment for us rule id 300700 'other bots' is being logging with errors for various API'S, i didn't see any information can anyone has more inputs on the existing issue will be helpful.

Azure Web Application Firewall

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. GitaraniSharma-MSFT 49,261 Reputation points Microsoft Employee
    2023-11-08T16:57:53.8533333+00:00

    Hello @rohith v ,

    Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.

    I understand that your Azure WAF is logging errors for various APIs with rule id 300700 'other bots' and you would like to know more details about the same.

    I discussed this internally with the Azure WAF Product Group team and below is the update on same:

    We won't be able to publish the internal definitions of our rules, as it would enable Bot creators to circumvent the rules. The bot manager rules are broken down into three categories as below:

    • BadBots (Bot100*) - Bot with malicious intent (bad IP reputation or modified user agents)
    • GoodBots (Bot200*) - Search engine traffic
    • UnknownBots (Bot300*) - Other bot user agents that may or may not be malicious.

    The Bot300* rules are just to inform the customer that the traffic is originating from a bot (non-browser traffic). The intent of the bot is unknown (it could be valid requests from users of the service, or it could be malicious attackers using a custom tool).

    Refer: https://video2.skills-academy.com/en-us/azure/web-application-firewall/ag/application-gateway-crs-rulegroups-rules?tabs=bot#bot-rules

    https://video2.skills-academy.com/en-us/azure/web-application-firewall/afds/waf-front-door-drs?tabs=bot#bot-rules

    Microsoft_BotManagerRuleSet-1.0-UnknownBots-Bot300700 - this is classified as an "unknown" bot which are published user agents without additional validation.

    If your requests are getting blocked by this rule ID, then you can check the details or message field of your WAF logs and see what is the matchVariableName and matchVariableValue triggered for those requests.

    Refer: https://video2.skills-academy.com/en-us/azure/web-application-firewall/ag/bot-protection-overview#log-example

    https://video2.skills-academy.com/en-us/azure/web-application-firewall/ag/web-application-firewall-logs#firewall-log

    In most cases, I've seen headers with "non-browser user agent" getting blocked by this rule.

    So, I would request you to go through your WAF rules and validate the requests and if you believe that legitimate traffic is getting blocked, you can create custom rules to allow that particular traffic.

    Refer: https://video2.skills-academy.com/en-us/azure/web-application-firewall/ag/web-application-firewall-troubleshoot

    Application gateway WAF custom rules: https://video2.skills-academy.com/en-us/azure/web-application-firewall/ag/custom-waf-rules-overview

    Azure Front Door WAF custom rules: https://video2.skills-academy.com/en-us/azure/web-application-firewall/afds/waf-front-door-custom-rules

    If you need help in understanding your WAF log, please share the particular WAF log and we can discuss about it further.

    Kindly let us know if the above helps or you need further assistance on this issue.

    Please "Accept the answer" if the information helped you. This will help us and others in the community as well.