What is bot management?

02How bots are detected

No single signal is reliable, so detection across the industry stacks many weak ones. Individually each has false positives; together they are hard to fake consistently. OWASP's bot management cheat sheet recommends combining controls at the edge, in the application and in the back end for the same reason.

03Fingerprinting, and its limits

A TLS fingerprint such as JA3 or JA4 summarises how a client negotiates the connection: the cipher suites it offers, its extensions and, in JA3's case, their order. It identifies the underlying library, which is why a basic script claiming to be Chrome stands out: it negotiates like its HTTP library, not like Chrome.

The limit is that fingerprints are shared, not unique. Every Chrome user on the same version and platform presents essentially the same handshake, so a fingerprint identifies a class of software, never a person. Tools also exist that imitate browser handshakes closely, so a match shows what a client presents, not what it is. Fingerprinting is good at answering 'what software is this?' and poor at answering 'what does it intend?'

04Responses, from lightest to heaviest

Bot management is not a block button. The response should be proportionate to the confidence and to the cost of being wrong.

• Allow: verified crawlers and known integrations, passed without friction
• Rate limit: tolerated automation kept to a sane pace rather than shut out
• Automatic challenge: a check the browser completes without the visitor doing anything, such as a proof-of-work challenge; a brief pause for a person, a real cost for a script running at volume
• Interactive challenge: a visible check such as a CAPTCHA for when an automatic one is not enough, with an accessible alternative for people who cannot complete it
• Block: for unambiguous cases such as exploit probes, known attack tooling and verified impersonators

05Matching the response to the threat

Each automated threat leaves different traces and calls for different controls. Several of the strongest controls live in the application rather than at the edge, which is why bot management works best when security and product teams agree on it together.

06Rolling it out without breaking things

Most of the damage bot management does comes from switching it on everywhere at once. A staged rollout avoids that.

• List the automation you depend on: search crawlers, uptime monitors, payment and webhook callbacks, partner APIs and your own mobile app
• Give machine clients an authenticated way in, such as API keys, signed webhooks or mutual TLS, instead of expecting them to behave like browsers
• Start in a log-only mode and review what would have been challenged; as OWASP's cheat sheet puts it, 'you cannot tune what you cannot see'
• Enforce first on the routes where abuse costs most: login, sign-up, checkout and search
• Watch business metrics as well as block counts: login success, conversion, basket abandonment and support tickets
• Keep an accessible alternative and a contact route for anyone who gets stuck

07The failure mode to design against

Every bot management system has a dial between catching bots and blocking customers, and the two are not equally expensive. Missing a scraper costs you some data. Blocking a customer costs you the customer, and they rarely tell you: they just leave.

Practical safeguards: corroborate before acting, prefer a challenge to a block where confidence is low, lift a client's penalties once it has proved itself, and review blocked traffic regularly for groups of legitimate users who share a trait, such as a school, an office network or an assistive technology.