Directive reference

mod_botshield registers 100 directives at config time. This page is the canonical reference, grouped by family. The underlying source-of-truth is bs_cmds[] in src/botshield.c:142 — when tuning behavior, treat the source as authoritative when it disagrees with a doc page.

Scope and validity

Most directives accept RSRC_CONF | ACCESS_CONF — they're valid in server config, <VirtualHost>, <Directory>, <Location>, <Files>, and their *Match variants.

A few are server-scope only (RSRC_CONF); placing them inside <VirtualHost> emits a NOTICE and the directive is ignored. The SHM segment is module-global, so any directive that sizes the segment or backs it with a state file lives at the main server level. These directives carry an explicit "(server scope only)" note in the table.

.htaccess is never valid for any BotShield directive — OR_ALL is never used. Bot-protection config in writable filesystem locations is a deliberate non-goal.

File-backed *File directives read their targets once at config- parse time and cache the bytes on the per-directory config — no per-request file I/O. Missing, unreadable, or oversized files fail apachectl configtest, so a broken template can't be reloaded into a running server.

Core

Directive	Syntax	Default
BotShieldEnabled	on|off|logonly	off
BotShieldChallenge	on|off	on
BotShieldDebug	on|off	off
BotShieldSecretFile	/path	unset (required)
BotShieldSecondarySecretFile	/path	unset
BotShieldAlgorithm	<name>	unset (required)
BotShieldCookieTTL	N (sec)	3600 (range 1..86400)
BotShieldCookieDomain	".example.com"	unset (host-only)
BotShieldDifficulty	N	3 (range 1..16)
BotShieldEndpointPrefix	/path	/botshield
BotShieldInteractiveArmMs	N (ms)	300 (range 0..2000)
BotShieldInteractiveMinSolveMs	N (ms)	400 (range 0..5000)

BotShieldEnabled is the master gate. It is tri-state:

• on — enforce. Tier decisions serve interstitials, triggers and rate-limit rules act on matches.
• off — module declines every request in scope; same shape as unloading the module on this path.
• logonly — observe-only. The handler runs and emits decision logs, but every enforcement-suppression site short-circuits: tier dispatch logs outcome=~challenge and declines instead of serving the interstitial; trigger / rate-limit matches log :observe and skip side effects. Use this to stage a whole policy revision before flipping enforcement on. See staging.

Because BotShieldEnabled is per-<Directory> / <Location>, operators can carve out exceptions:

BotShieldEnabled LogOnly                # vhost: observe
<Location "/about">
    BotShieldEnabled On                 # /about: enforce
</Location>

BotShieldChallenge Off makes a scope block-only: triggers, rate limits and scoring all still run and still log, but no interstitial, form or captcha is ever rendered — any selected tier collapses back to pass, and the suppression appears in the decision log as challengeoff:<tier>.

Use it where an explicit respond=4xx trigger is meant to be the only action. It is the only thing that holds: a flag tier_floor and a BotShieldChallengeAtLeast row are both MAX'd in at the tier decision, so silencing the things that feed them is not the same as silencing them. BotShieldChallenge Off is applied after the floor, so it does what the others only appear to.

This has bitten a production deployment, so it is worth spelling out what the ceiling does and does not buy. A hub running only the silent tier parked Hard and Captcha at 10000, believing form and captcha were unreachable. They were not: a flag was forcing a tier directly, bypassing the thresholds entirely.

At the time those floors were seeded in and the config said nothing about them, which is what made the surprise possible. Nothing is seeded now, and a flag reaches a tier only through a rule you wrote:

<BotShieldRule escalate-honeypot>
    BotShieldFlagged   honeypot_hit
    BotShieldChallenge captcha
</BotShieldRule>

BotShieldChallenge on a rule sets a floor, and a floor is "at least this intense regardless of cumulative score" — so it opts that flag out of your ceiling, by design. Writing one is the deliberate act; the hazard is only that it used to happen without you.

Score instead of a floor if you want the ceiling to keep applying:

<BotShieldRule score-honeypot>
    BotShieldFlagged   honeypot_hit
    BotShieldScore     botsignals +60
</BotShieldRule>

That contributes to an accumulator a BotShieldChallengeAtLeast row reads, so it is subject to whatever thresholds you set.

BotShieldDebug returns 403 "Hello World" for every request in scope — useful as a smoke test that the hook is firing.

BotShieldSecretFile and BotShieldAlgorithm are required. The module emits 503 X-Botshield: misconfigured for any request where both are not resolved on the scope. BotShieldSecondarySecretFile is the verify-only secondary key for graceful rotation — see deployment.

BotShieldAlgorithm only sha256zeros is built-in today; sha384-zeros, sha512-zeros, pbkdf2-sha256, argon2id are reserved registry slots that fail with a clear "not implemented" diagnostic.

BotShieldCookieDomain adds a Domain= attribute to Set-Cookie so reputation follows across subdomains. Default is host-only. When HTTPS is in use AND no domain is set, the module emits the __Host-bs_session cookie name; otherwise the legacy _bs_session. Verify path checks both.

BotShieldDifficulty is the leading-hex-zeros count for the PoW, so expected work is 2^(4*d) hashes: 3 is 4,096, 4 is 65,536. Each step up multiplies client cost by 16, not by a little.

Measured on live traffic rather than estimated. At 4, 905 real solves cost a median 329ms of client CPU, p90 1019ms, p99 5188ms, and 34% of visitors spent over half a second; Android was worst at a 733ms median. At 3 the same population runs a ~50ms median. The default is 3.

Raising it is rarely worth it. Proof-of-work here is a capability test — it proves a real JS engine ran — not an economic barrier. A native solver is 15–43× faster than a browser at the same difficulty, so cost falls on your visitors far harder than on a competent bot, and a cookie amortises the bot's one solve across the whole BotShieldCookieTTL window. Of those 905 solves, 2 carried any failed attestation probe: the population that runs the JS is already clean.

There is no master switch for the implicit path. BotShieldScoring used to be one, and it was removed: the module now ships no default rules at all, and an unset score threshold never fires, so nothing scores and score never decides a tier until an operator writes both a heuristic or flag trigger and a threshold.

That removal is the same lesson the directive was a workaround for. Every lockout this module has caused came from an implicit weight or tier floor nobody had written down summing past a threshold nobody had read. A switch to turn that off is worse than not inventing it: a default every deployment has to disable was not a default. An explicit rule states what it does; a score states it only once you reconstruct the arithmetic.

BotShieldInteractiveArmMs withholds the interactive tier's checkbox for this many milliseconds after load (0 shows it immediately). The widget renders at full size with a spinner meanwhile, so nothing shifts and there is no dead control to click — a visible control that swallows clicks teaches people the page is broken, and it is the confident fast clickers who hit it.

The delay is not the point; the fact that the delay is ours is. A bare time floor is a guess about how fast people are and it charges the quick ones. Withholding the control makes "window + reaction" a fact, so the server can floor at least the window for free. It also yields a sharper signal: a person clicks hundreds of ms after the reveal, a poller within a few, and that gap is reported as an attest: failure. 100 is a good production value — below the threshold where a delay registers at all, while the gap signal is unaffected by the shrink.

BotShieldInteractiveMinSolveMs refuses an interactive-tier solve that returns sooner than this after the challenge was issued (0 disables). The clock is entirely the server's — the issue stamp rides in the bootstrap HMAC — so a client cannot shorten it.

Measured, not guessed. A warmed headless browser needed 177ms to load, render and land a trusted click, and 239ms cold; the first version of this floor was 150ms, which sat under both and therefore cost an attacker nothing. Note what it does and does not do: it will not stop a bot that sleeps, and a cold browser is slow enough to clear any sane floor on its own. What it does is make waiting mandatory and server-observed, so the delay is real wall time per request rather than a field the client fills in. Watch reason=solve_too_fast for false positives.

BotShieldEndpointPrefix is the URL prefix for module-owned handlers (/botshield/captcha-verify, /botshield/metrics, /botshield/embedded.js, /botshield/safeguard-info etc.).

Under that prefix, <prefix>/preview is an index of the pages a client can be shown, rendered from the live templates rather than from copies that drift:

Route	Shows
<prefix>/preview/noninteractive	The self-solving widget, pinned mid-check (the preview's proof-of-work is set unsolvable so it cannot complete and navigate away)
<prefix>/preview/interactive	The same widget with a live checkbox waiting for a click
<prefix>/preview/safeguard	The anti-loop explainer, also served at <prefix>/safeguard-info, which is the URL clients actually reach

They mint nothing and are useful for design review and for seeing what a visitor sees without having to trip a challenge yourself. They serve 403, like the real interstitials, because the same code path renders them. Change it if it collides with real app routes.

What used to be here

The three BotShieldScore* cut-points and the four BotShieldForgiveness* directives. The cut-points turned one running total into a tier; forgiveness subtracted from that total when a client solved, with an hourly cap so a patient bot could not farm credit by solving cheap challenges.

Both are gone, and the total they operated on is no longer carried in the cookie. A tier is chosen by a rule that asks for one, or by a BotShieldChallengeAtLeast row reading a named accumulator — both of which name the signal that paid for the challenge, which a single running total never could.

What forgiveness was protecting is still protected, at the tier decision rather than in the score: a client is not challenged at a tier it has already passed. Forgiveness could never break a challenge loop on its own, because flag effects re-apply every request — a forgiven-to-zero score was re-raised on the next one. See site model.

Silent-tier dispatch

Directive	Syntax	Default
BotShieldNonInteractiveMode	interstitial|embedded	interstitial

interstitial (the default) serves a no-click splash page that auto-submits a SHA-256 PoW on load — the legacy noninteractive-tier behavior. embedded instead hands off to the site-included /botshield/embedded.js wrapper: the page serves DECLINED (real content) and the wrapper does the PoW in a Web Worker, then POSTs the result back to /botshield/embedded-verify to mint _bs_session on the next request. Embedded mode trades a brief window where the cookie isn't yet on the client (the very first request goes through unverified) for a zero-interstitial UX.

Embedded mode requires you to include the wrapper script in your page templates; without it, the request still serves the real content but no cookie ever lands.

Widget customization

Directive	Syntax	Default
One container, <BotShieldChallengePage>, once per scope. It takes no
argument and describes how the interstitial looks.

<BotShieldChallengePage>
    BotShieldPrompt     "Please verify"
    BotShieldNotice     "Checking your browser…"
    BotShieldLogoFile   /etc/botshield/logo.svg
    BotShieldLogoLabel  "Crestline"
    BotShieldShowLogo   On
    BotShieldHelp       button
    BotShieldHelpFile   /etc/botshield/help.html
</BotShieldChallengePage>

Inside the container	Syntax	Default
BotShieldPrompt	"text"	I'm not a robot
BotShieldNotice	"text"	Verifying you are human…
BotShieldLogoFile	/path.svg	embedded Guardian
BotShieldLogoLabel	"text"	botshield
BotShieldShowLogo	On|Off	On
BotShieldShowLabel	On|Off	On
BotShieldShowBox	On|Off	On
BotShieldHelp	off|on|button	button
BotShieldHelpFile	/path.html	built-in text
BotShieldTemplate	/path.html	built-in shell

BotShieldPrompt and BotShieldNotice are not the same string. The interactive and captcha tiers put a control in front of the client, so their line is an invitation. The non-interactive tier gives them nothing to do, so its line is a status — and writing an invitation there asks a question the page offers no way to answer. Set whichever you customise; the other keeps its built-in.

Before 2026-09-11 a single BotShieldPrompt won on every tier, so setting it silently replaced the status too. If you set a prompt and want that old text on the self-solving tier as well, write it into BotShieldNotice explicitly.

This is the page every challenged request sees, not the captcha tier's alone — bs_render_challenge_page draws all three tiers and the captcha widget is one branch inside it. That is why these settings are not part of <BotShieldCaptcha>: a scope that configures no captcha still renders this page, and making them per-provider would mean declaring a captcha block just to set a logo.

Per-directory like <BotShieldCaptcha>, so a <Location> can dress its own interstitial.

Moved: the nine flat spellings

BotShieldPromptText, BotShieldLogoFile, BotShieldLogoLabel, BotShieldShowLogo, BotShieldShowLabel, BotShieldShowBox, BotShieldHelp, BotShieldHelpFile and BotShieldChallengeFile moved inside the block on 2026-09-11 and now fail config parse with a line naming where the setting went. Two changed name in the move: BotShieldPromptText → BotShieldPrompt, and BotShieldChallengeFile → BotShieldTemplate, which is what a full HTML page carrying the widget marker always was.

BotShieldTemplate replaces the full HTML page that wraps the widget; the file must contain <!-- BOTSHIELD --> where the widget is spliced in. Other widget directives still apply to the widget block itself. Max 256 KiB.

Logo and help files are 64 KiB max each. Logo content is served inline as <img>-equivalent SVG; help content is rendered as trusted HTML (no escaping — you own sanitization).

BotShieldShowLogo / ShowLabel / ShowBox strip widget chrome down to a lone checkbox if the surrounding page styles its own chrome. When label is hidden it moves to the button's aria-label — accessibility is preserved.

Captcha tier

One block per provider, the provider as its argument:

<BotShieldCaptcha turnstile>
    BotShieldSiteKey           0x4AAAAAAA
    BotShieldSecretFile        /etc/botshield/turnstile.secret
    BotShieldExpectedHostname  example.org
    BotShieldExpectedAction    botshield
    BotShieldCABundle          /etc/ssl/certs/ca-bundle.crt
</BotShieldCaptcha>

BotShieldCaptchaTimeout        1000
BotShieldCaptchaConnectTimeout 250
BotShieldCaptchaRateLimit      30
BotShieldCaptchaMaxInFlight    64

Inside the block	Meaning
BotShieldSiteKey	provider-public key embedded in the widget
BotShieldSecretFile	siteverify secret; must be mode 0600
BotShieldExpectedHostname	hostname the provider must echo back; empty disables
BotShieldExpectedAction	action the widget must tag the token with; empty disables
BotShieldCABundle	PEM bundle for the provider's TLS certificate
BotShieldMinScore	reCAPTCHA v3 only: minimum accepted score

A scope may hold several blocks. The first declared is the scope's provider -- what the interstitial renders and what the bare <prefix>/captcha-verify checks against. The rest are reachable two ways: <prefix>/captcha-verify/<name> verifies with the named block's secret, and a rule can name one to render:

<BotShieldRule suspected-scraper>
    BotShieldUserAgent  @bot
    BotShieldChallenge  captcha hcaptcha
</BotShieldRule>

That is friction by evidence -- an invisible widget for a borderline human, a harder one for a client the module already doubts. Only the captcha tier takes a provider; BotShieldChallenge interactive hcaptcha is refused, because neither of the other tiers renders one.

An unknown provider name is refused at config time against the compiled-in registry. Whether the scope configures it cannot be checked there -- rules are server scope and a block may be per-<Location> -- so a name the scope has no block for falls back at render time to the scope's own provider, exactly as a single-provider scope has always behaved.

Form captcha and the embedded path keep the scope's provider whatever a rule says: their token round-trip carries no provider name, so there would be nothing to verify the answer against.

The provider is the block argument because it is the value that decides whether the rest mean anything -- a site key without a provider configures nothing -- and it names the block in every error the settings inside can raise. RSRC_CONF | ACCESS_CONF, so a <Location> may declare its own; one per scope, and a second is refused rather than letting the later one win silently.

Four captcha directives stay outside it. BotShieldCaptchaTimeout, BotShieldCaptchaConnectTimeout, BotShieldCaptchaRateLimit and BotShieldCaptchaMaxInFlight are scope defaults: a vhost sets them once and every provider block under it inherits. The first two are network timeouts, the last two protect the verify endpoint; none describes a provider, and moving them in would make a per-<Location> provider block the only place to say something that is not per-provider.

BotShieldMinScore replaces BotShieldRecaptchaV3MinScore and is refused under any other provider. As a top-level directive it parsed anywhere and silently did nothing for five of the six.

Until 2026-09-10 these were seven standalone directives (BotShieldCaptchaProvider, ...SiteKey, ...SecretFile, ...ExpectedHostname, ...ExpectedAction, ...CABundle, BotShieldRecaptchaV3MinScore).

Directive	Syntax	Default
BotShieldCaptchaTimeout	N (ms)	1000 (100..5000)
BotShieldCaptchaConnectTimeout	N (ms)	250 (50..5000)
BotShieldFormCaptcha	on|off	off

Provider names: turnstile, hcaptcha, recaptcha-v2, recaptcha-v3, friendly, geetest. See captcha for the wire-protocol details.

BotShieldCaptchaTimeout is the total siteverify HTTP budget; on timeout the verify path fails open. BotShieldCaptchaConnectTimeout is the connect phase only — tighter, raised on links with transient packet loss.

BotShieldMinScore only matters for recaptcha-v3, which is why the block refuses it anywhere else. Reject verifications below this score even on success: true.

BotShieldExpectedHostname / BotShieldExpectedAction: empty string disables the check; unset uses defaults (server_hostname / "botshield"). GeeTest binds host/action via HMAC and doesn't return them in the response, so these are no-ops for that provider.

BotShieldFormCaptcha intercepts POSTs and validates the captcha token inline rather than via interstitial. Requires a captcha provider configured on the same scope.

Captcha-verify endpoint hardening

Directive	Syntax	Default	Scope
BotShieldCaptchaRateLimit	N (per IP per minute)	30 (0..1000)	server / vhost
BotShieldCaptchaMaxInFlight	N (global concurrent)	64 (1..1024)	server only

Both reject before any libcurl call. RateLimit returns 429 with Retry-After; MaxInFlight returns 503. 0 on RateLimit disables.

SHM sizing and persistence

All directives in this section are server scope only. Inside <VirtualHost> they emit a NOTICE and are ignored.

Directive	Syntax	Default
BotShieldShmSize	<size> (128K..256M)	16M
BotShieldFlaggedIPCapacity	N	50000 (1024..1000000)
BotShieldForgetIPAfter	N (sec)	3600 (1..2592000)
BotShieldIPv6PrefixLen	N (0..128)	64
BotShieldBloomIPs	N	1000000 (1000..10000000)
BotShieldBloomWindow	N (sec)	604800 (3600..2592000)
BotShieldDataDir	/path	/var/lib/botshield
BotShieldDbStatsFile	/path	/run/botshield/db-load.stats
BotShieldFpmStatsFile	/path	/run/botshield/fpm-load.stats
BotShieldStateFile	/path	<datadir>/state.bin
BotShieldStateSaveInterval	N (sec)	300 (0=shutdown-only)
BotShieldEscalateCapacity	N	50000
BotShieldSafeguardCapacity	N	50000
BotShieldEmbeddedNonceCapacity	N	32768 (1024..1048576)

BotShieldShmSize is the total budget for the flagged-IP / strike / safeguard tables and the Bloom buffers. BotShieldFlaggedIPCapacity etc. size individual tables within that budget — the headroom watchdog will flag if the segment is underprovisioned.

BotShieldIPv6PrefixLen masks IPv6 client IPs before keying the SHM tables. Default /64 is per-subscriber for typical ISP allocations; tighter values (/56, /48) flag larger blocks of addresses sharing reputation.

BotShieldStateFile gives crash-durable persistence to the flagged-IP table, the Bloom filters, and the metrics counters and dashboard bucket rings. Rate-limit counters are deliberately excluded — see Deployment. The periodic save requires mod_watchdog; the graceful-shutdown save runs regardless. State format mismatches on load reject the file with a NOTICE and start fresh — never a startup failure.

It defaults to <BotShieldDataDir>/state.bin wherever the module is enabled on any vhost. Without a state file every restart — including one triggered by logrotate — silently empties the flagged-IP table, the Bloom filters and every dashboard counter, and nothing in the output says so. That is a default that enables no enforcement and changes no decision; it only stops the module forgetting what it already learned.

BotShieldDataDir and running two instances

BotShieldDataDir is the one directive a second httpd instance on the same host has to set. Everything the module owns on disk lives in it:

File	What it is
secret	the auto-generated cookie-signing key
state.bin	flagged-IP table, Bloom filters, counters

Both default into /var/lib/botshield, so two instances that leave it alone share both. Sharing the state file loses reputation data; sharing the secret is worse, because whichever instance writes it last invalidates every cookie the other has issued. Give the second one its own:

BotShieldDataDir /var/lib/botshield/instance2

The directory is created at startup — during post-config, the one moment the module runs as root — and chowned to the Apache user, which is required rather than tidy: the periodic state save runs in a child after the privilege drop, while the shutdown save runs in the root parent. It is created recursively, so nesting under the default works with the parent absent. If it cannot be created the module logs a NOTICE and starts cold; a convenience default is never a reason to refuse to start.

Deriving a per-instance name automatically was tried and rejected. ServerRoot and DefaultRuntimeDir are both unusable: instances routinely share a ServerRoot, and the runtime dir is ephemeral and reset at boot. The listen set does work, being the one thing the OS forces two instances to differ on, but it changes whenever an operator edits Listen, which would rename the files and start that instance cold. A default that quietly moves is worse than one you declare.

The two stats paths default to what the sidecars in tools/ publish: botshield-dbmon.service passes --state-file /run/botshield/db-load.state and the monitor derives its .stats companion from that name, so the two halves are one shipped convention. Installing the units is therefore enough to light up the dashboard's load graph, with no directive. A missing file is the normal case for a deployment without the sidecars and costs one failed open per watchdog tick; the graph simply stays empty.

Unlike BotShieldDataDir these are read-only and operator-published, so two instances sharing them is correct — they are reading the same host's telemetry.

These are all server scope, and enforced

Every directive in the table above is rejected inside <VirtualHost>, with an error naming the directive. They are resolved once from the main server: the SHM segment is sized and attached before vhosts merge, and the load watchdog is registered against the main server_rec. A copy on a vhost is parsed and then never read.

This used to be a startup NOTICE, which was not enough. The failure it produces is invisible in the shape operators actually check — the config parses, configtest is green, httpd starts, and the directive simply does nothing. BotShieldDbStatsFile in a vhost left the dashboard reporting "no monitor" behind a clean configtest. RSRC_CONF permits vhost context, so Apache will not catch it; the module has to.

UA classification and allow list

Each request gets one User-Agent classification, computed once and cached for every downstream consumer. It composes four passes: real-browser templates, the knownbot directory, verified-bot IP cross-check, and a heuristic scan for bot-shaped UAs matching nothing else.

Directive	Syntax	Default	Scope
BotShieldClassify	on|off, or [all|none] [+/-<pass>]...	all four passes on	server / vhost
BotShieldAllowBot	<name> <ua-pattern> [<target>]	builtin only	server / vhost
BotShieldAllowRangesRefreshInterval	N (sec, 0..86400)	0 (disabled)	server / vhost
BotShieldBotDirectory	/path (TSV)	unset (compiled-in baseline)	server / vhost
BotShieldBrowserTemplates	/path (text)	unset (compiled-in baseline)	server / vhost
BotShieldDataRefreshInterval	N (sec, 0..86400)	300; 0 selects that default, negative disables	server / vhost

BotShieldClassify toggles individual passes — browsers, known-bots, verified-bots, unknown-bots. Mixing the two grammars is a config-time error:

BotShieldClassify Off                   # standalone form, one token
BotShieldClassify All -verified-bots    # compositional form
BotShieldClassify None +browsers        # start from nothing, add back

Each disabled pass has a fail-safe rather than a silent behavior change: -browsers treats every UA as a browser for robots.txt wildcard purposes; -known-bots skips the directory walk, so no bot slug reaches the log; -verified-bots skips the IP cross-check and matched UAs degrade to knownbot, so they answer neither ua=@verified-bot nor ua=@fake-bot and any rule written on those stops applying to them (the intended response to stale CIDR data); -unknown-bots skips the bot-token substring scan.

BotShieldAllowBot registers a UA pattern + IP-range pair. The third arg is a path to a CIDR file, a single CIDR, comma-separated inline CIDRs, or * alone for UA-only matching (logged as knownbot:<name>, score 0); omit it for /var/lib/botshield/bots/<name>.txt. A <name> matching a bundled built-in (googlebot, bingbot, applebot, googleother, siteimprove) replaces that built-in. See policy for the verified / fake / unverified outcomes.

BotShieldAllowRangesRefreshInterval re-stats the verified-bot CIDR files (<name>.txt plus an operator sidecar <base>.local.txt) and atomic-swaps a rebuilt range set on any mtime change. The sidecar is the supported seam for scanner IPs absent from a vendor's public feed. Recommended 60-300 when enabled; 0 keeps the config-time load and needs a graceful restart to pick up edits.

The two data-source pairs override their compiled-in baselines without a rebuild — refresh the files with services/refresh/botshield-refresh.py directory and services/refresh/botshield-refresh.py user-agents, and the watchdog re-parses on mtime change. If a file disappears or fails to parse, lookups fall back to the baseline codegenned into the .so at build time. BotShieldBotDirectory is TSV (pattern|slug|category|followsRobotsTxt, # comments); BotShieldBrowserTemplates is one normalized UA template per line (runs of [0-9._]+ replaced by X).

Rate limit

Directive	Syntax	Default
BotShieldBotRateLimit	off, or <target> <delay-sec>, or <target> <budget> <per>	none
BotShieldEscalate	<rule> <strikes> <per> [respond=N] [ttl=N]	none (BotShieldRateLimitEscalate until 2026-09-09)

Alternation on ua=

ua= accepts a comma-separated list of @selectors, matching if any one does:

<BotShieldRule api-bot>
    BotShieldPath         /api/*
    BotShieldUserAgent    @search,@ai-input,@ai-train,@monitor
    BotShieldRespond      403
    BotShieldLogAs        api-bot
</BotShieldRule>

That replaces four rules identical but for a single token. Expanded at parse time into one entry per alternative — this family is strict first-match-wins, so N adjacent entries differing only on the UA axis and carrying identical actions are exactly equivalent to one entry with an OR. Copies take a #N name internally; the decision log is unaffected, since it reports the action's logas= tag, which every copy shares.

Only @selectors split on one line. A bare substring pattern written as a single value is passed through untouched, because a User-Agent legitimately contains commas — Mozilla/5.0 (X11; Linux x86_64) is full of them — and splitting those would silently change what an existing rule matches.

Repeating the directive ORs, whatever the values are. Inside a block, BotShieldUserAgent accumulates the way BotShieldPath does — one alternative per line, no separator to be ambiguous about:

<BotShieldRule mixed>
    BotShieldUserAgent  @ai-train
    BotShieldUserAgent  CorpBot/1.0
    BotShieldUserAgent  Mozilla/5.0 (X11; Linux x86_64) Grabber/2
    BotShieldRespond    403
</BotShieldRule>

All three are alternatives, and the third stays one value despite its commas. This is the spelling to reach for when the alternatives are not all @selectors; the one-line comma form remains available and is still the shorter way to write a list of selectors.

Until 2026-09-07 repeated lines were comma-joined before parsing, so two plain values became the single literal substring CorpBot/1.0,OtherBot/2.0 — a rule that matched nothing, with no warning, because repetition was accepted and only the @selector split could undo the join.

BotShieldRateLimit is retired (2026-09-09)

It was a name, a cohort (ua= and ipspec=), a budget and a window. A rule has all of that and everything else -- a path, a query, a cookie, a flag -- so the directive was a rule that could say less:

# then:  BotShieldRateLimit scrapers 10 min ua="wget"
# now:
<BotShieldRule scrapers>
    BotShieldUserAgent  wget
    BotShieldRate       10 60
</BotShieldRule>

Same reason (ratelimitexceeded:scrapers), same 429 with Retry-After, same +50 score, same flag on the address, and BotShieldEscalate scrapers ... binds to the rule by name exactly as it bound to the directive. mode=observe on the directive is BotShieldMode observe on the rule, and a scope-level BotShieldEnabled LogOnly observes a rule's window as it observed the directive's. Nothing was lost in the move except the second spelling.

BotShieldBotRateLimit caps volume per knownbot slug rather than per cohort. <target> is a bot slug or UA substring resolved against the bot directory, @<botgroup> (search, ai-input, ai-train, monitor), or *. The two-arg form is Crawl-delay shaped — one request per <delay-sec>, where 0 admits everything. Precedence is specific > @botgroup > *.

* pre-allocates one counter per directory slug not covered by a more specific rule, so each unmatched bot gets its own budget rather than sharing one; three reserved aggregate slots at the same budget cover unknownbot, fake-bot, and slugs a mid-run directory refresh added after startup. Because the slug universe is bounded by the directory, every slot is allocated at config time — request time is a single hash probe. Over-budget returns 429 + Retry-After with reason botrate:<slug>. Robots.txt Crawl-delay groups feed the same machinery.

Nothing is rate limited until you say so

There is no default. An unset BotShieldBotRateLimit rate-limits nothing, the same as every other policy family in this module.

It used to work the other way. Enabling the module with no BotShieldBotRateLimit anywhere synthesised a wildcard of one request per second per directory slug, and the synthesised entry carried no mode, which means enforce. The result was real 429s to traffic nobody had decided to throttle, including named crawlers arriving with a published identity. It was invisible twice over: absent from the config because nobody wrote it, and absent from the policy dump because the dump only knew about directives. On the deployment this was built for it was returning 429s to Applebot, GPTBot and Claude-User, and the way it came to light was reading a decision log, which is not how an operator should have to discover what their server is enforcing.

Two sources now, both of which an operator can point at:

BotShieldBotRateLimit * 1 sec mode=observe   # count, refuse nobody
BotShieldBotRateLimit @ai-train 10 min       # a real limit
BotShieldBotRateLimit off                    # disable the subsystem

and a Crawl-delay in a configured robots.txt, which feeds the same machinery.

off declines robots.txt Crawl-delay groups, which is the only implicit source left now that the default is gone. Rules written in the config still apply: off never meant "ignore what I explicitly asked for". It exists because a robots.txt is frequently maintained by somebody other than whoever configures this module, and an operator has to be able to refuse a Crawl-delay appearing in a file they do not own.

httpd -t -D DUMP_BOTSHIELD_POLICY lists every directive-derived entry with its budget, window, scope, and whether it enforces or observes. Crawl-delay entries register after that dump runs, so they appear in its robots.txt section rather than this one.

Why BotShieldBotRateLimit stays (2026-09-09)

Everything it can say a rule can now say: scope=each is BotShieldDelay or BotShieldRate <n> <s> each, scope=group is a rule with BotShieldUserAgent @group and BotShieldRate, and scope=total is the same with @bot. Rules also spend in sequence, so a per-crawler rule above a shared-group rule above a ceiling rule is the tier stack the directive built by hand.

It stays because the bots page is fed from its request-time check: the per-bot request totals and the User-Agent sample the dashboard shows come from the slot the wildcard gives every directory slug. A rule's per-crawler slots are the same counters but do no such accounting, and the dashboard reads only the directive's table. Retiring the directive means moving that accounting into the rule path and the dashboard onto rule slots -- worth doing, and its own piece of work rather than a footnote to this one. Until then a BotShieldBotRateLimit * 1 sec is the line that keeps the bots page populated, whatever else rate-limits.

Robots.txt enforcement

One container, <BotShieldRobots>, once per server scope. Everything robots.txt enforcement can be told lives inside it: the file, the mode, how User-agent: * is applied, the refresh cadence, and any groups written directly in the config.

<BotShieldRobots>
    BotShieldRobotsTxt        /etc/botshield/robots.txt
    BotShieldMode             observe
    BotShieldWildcardScope    heuristic
    BotShieldRefreshInterval  60

    <BotShieldRobotRule ai-crawlers>
        BotShieldUserAgent   GPTBot
        BotShieldUserAgent   ClaudeBot
        BotShieldDisallow    /
        BotShieldRespond     404
        BotShieldLogAs       ai-deny
        BotShieldMode        enforce
    </BotShieldRobotRule>
</BotShieldRobots>

Inside the container	Syntax	Default
BotShieldRobotsTxt	/path	none -- the container may be groups only
BotShieldMode	enforce|observe	enforce
BotShieldWildcardScope	heuristic|strict|off	heuristic
BotShieldRefreshInterval	N (sec)	60 (0 disables; needs a file)
<BotShieldRobotRule name>	a group; see below

Inside a group	Syntax	Notes
BotShieldUserAgent	product token, *, or @botgroup	repeatable; at least one
BotShieldDisallow / BotShieldAllow	/pattern (* and $ as RFC 9309)	repeatable
BotShieldCrawlDelay	seconds, fractions allowed	one request per window, per crawler
BotShieldRespond	4xx or 5xx, not 429	what a Disallow answers; default 403
BotShieldMode	enforce|observe	overrides the container's for this group
BotShieldLogAs	tag	rides the decision line as tag="..."

The container is the precedence scope. The file's groups and the inline groups are one document, and robots.txt precedence -- the most specific User-agent group wins, the longest path rule within it, an Allow on a length tie -- is computed across all of them together. A named inline group therefore suppresses the file's User-agent: * for its crawler exactly as a named group in the file would. That is why there is one container per scope and a second is refused: two would be two independent sets, and a rule's meaning depends on its siblings.

Mode is inherited and a group may override it. Both directions are useful: an observing container with one group armed is how enforcement gets staged one crawler at a time; an enforcing container with one group observing is how a new group joins. When an observe group's Disallow is the longest match, it records what it would have done (robotsblock:<group>:observe, outcome=~block) and steps aside -- the longest enforcing rule still applies. An observed rule never shadows an enforced one, which is what BotShieldMode observe already means on a request rule.

Observe never enforces anything: no status, no +100, no robots_ignored flag. A scope-level BotShieldEnabled LogOnly observes every group, enforce overrides included.

BotShieldRespond is what a Disallow answers. Crawl-delay trips always answer 429 with Retry-After, because that is a request to come back later rather than a refusal, so BotShieldRespond 429 is rejected rather than accepted as a Disallow that invites a retry.

httpd -t -D DUMP_BOTSHIELD_POLICY lists every group with its source (file or config) and any knobs it set.

Until 2026-09-09 this was four standalone directives -- BotShieldRobotsTxt, BotShieldRobotsMode, BotShieldRobotsWildcardScope, BotShieldRobotsRefreshInterval -- and the file was the only place a policy could be written. The container folds them and adds what the file cannot say: a status code, a per-group mode, a log tag, and a group that exists in the config alone.

See policy for the matcher semantics and refresh model.

Triggers

Directive	Predicate args	Action keys
BotShieldRule	<name> + any of path=<glob> query=<glob> cookies=none|any|session ua=<substring>|@<botgroup>|"" ipspec=<spec> — ANDed, at least one required	respond=, redirect=, logas=, accesslog=, flagip=, flagsession=, score=, mode=
BotShieldFeedback	<label> + BotShieldEvent	flagip=, flagsession= (both accept +/-/=), logas=, accesslog=, mode=
BotShieldSessionCookieName	<name> (single arg, repeatable)	n/a (feeds cookies=session predicate)

See policy for full predicate vocabulary and family-by-family semantics.

BotShieldSessionCookieName registers a name that the cookies=session cookietrigger predicate considers a session cookie. Repeatable; each call appends.

accesslog=off — keep a request out of the access log

logas=<tag> names a tag that rides the decision line as tag="<x>" for fail2ban handoff. accesslog=off is separate and independent: it suppresses the access-log line for a matching request.

They compose, which is the point — a scanner probe usually wants both:

<BotShieldRule env-probe>
    BotShieldPath         /.env
    BotShieldRespond      403
    BotShieldLogAs          scanner-probe
    BotShieldAccessLog    off
</BotShieldRule>

(An earlier development build overloaded logas=off for this. That form is now rejected at config time with a pointer here, rather than silently being treated as a tag named "off" and losing the suppression.)

The mechanism is Apache's own. log_transaction is a RUN_ALL hook declared with (OK, DECLINED), so a hook returning any other value breaks the chain. The module's bs_propagate_decision_env hook runs at APR_HOOK_FIRST — ahead of mod_log_config — and returns DONE when the marker is set. Nothing downstream ever formats or writes a line.

Three consequences to understand before using it:

• A CustomLog-based decision log is suppressed too. mod_log_config services every CustomLog from a single hook function, so there is no way to keep one and starve another. BotShieldDecisionLog is written by the module itself and is unaffected, as is the error-log line — pair the two and you get the useful split: flood traffic out of an archived access log, every decision still recorded in a separately-rotated detection log.
• Third-party loggers are also skipped if their log_transaction hook is ordered after this module's (audit logging, analytics).
• It never applies under mode=observe or BotShieldEnabled LogOnly. The action engine returns before any side effect in that path, so a dry run always leaves its evidence.

If you want a request dropped from one specific CustomLog while keeping the others, this is the wrong tool — use mod_log_config's own conditional form, which can key off the decision the module already published to the request environment:

CustomLog logs/access.log combined "expr=%{reqenv:BS_OUTCOME} != 'block'"

That is the right tool when you only want to thin one log.

BotShieldMatch — name a set of conditions and reuse it

Directive	Syntax	Scope
<BotShieldMatch <name>>	conditions, one per line, until </BotShieldMatch>	server / vhost
BotShieldMatches <name>	inside a rule — splices that set's conditions in

A set holds the predicate half of a rule, lifted out so more than one rule can share it. It is expanded textually at config time, so a rule naming a set is exactly the rule you would have written by hand: no family, parser or evaluator downstream knows that sets exist.

<BotShieldMatch gated-content>
    BotShieldPath  /$
    BotShieldPath  /search/
    BotShieldPath  /publications
</BotShieldMatch>

<BotShieldRule crawler-pass>
    BotShieldMatches   gated-content
    BotShieldCrawler   yes
    BotShieldNoChallenge
</BotShieldRule>

<BotShieldRule content-gate>
    BotShieldMatches   gated-content
    BotShieldSolved    no
    BotShieldChallenge noninteractive
</BotShieldRule>

The case this is for. Those two rules are a matched pair — one exempts declared crawlers from the gate, the other gates everyone else — and written out longhand they repeat the same path list twice with nothing saying they belong together. Add a path to one and forget the other and verified crawlers start being challenged on it, which is the failure the tier_floor warning above describes from the other direction. A name is how the config says these two are about the same thing.

Conditions only. Action keys are refused inside a set: a block named for what it matches must not also decide what happens, or every rule naming it inherits an action from somewhere else in the file. Put the action on the rules.

Defined before used, and only once. Resolution happens where the BotShieldMatches line is read, so a set must appear above the rules that name it; naming an undefined set is a config error rather than a rule that quietly matches nothing. Defining the same name twice is also refused — last-one-wins is precisely how a shared set stops being shared.

Two things follow from resolving in definition order: a set may name a set defined above it, and a cycle cannot be built, because a name that is not yet defined does not resolve.

A vhost inherits sets from server scope and may shadow one by name, the same rule BotShieldAllowBot follows.

BotShieldRule — match on any request property

Every match key is optional and they AND together. At server scope at least one is required: a rule with no condition matches every request, which nobody writes on purpose. Inside <Location>, <Directory>, <Files> or <If> none is required — Apache has already matched, and the container is the condition. See policy.

Key	Matches against	Notes
path=<glob>	r->uri — path only, no query string	must start with /, ≤256 chars
query=<glob>	the query string alone	e.g. query="*return=*"
cookies=none|any|session	the parsed Cookie header	bulk forms only
ua=<substring>|@<selector>[,@<selector>...]|""	User-Agent	* means "any", same as omitting. ua="" matches a request with no User-Agent header, or one present but empty — absence is not a substring, so it needs its own spelling. A comma list of @selectors matches if any of them does.
acceptlanguage=""|*	the Accept-Language header	"" = absent or empty, * = present. The missingal signal as a condition; deliberately not a header matcher
scoreatleast=<name> <n>	a named per-request accumulator	matches when rules above this one have moved <name> to at least <n>
ipspec=<spec>	client IP	*, a CIDR list, or a file path
solved=yes|no	authenticated solve proof in the cookie	the strongest single predicate here: on a production hub 100% of challenges carried "no solve proof"
exists=yes|no	whether the request maps to a real file	a stat, so the rest of a ladder can see only paths that do not exist
crawler=yes|no	verified-bot classification	identity-checked, not UA text
firstsight=yes|no	Bloom-filter membership of the address	yes = never seen before
loadavgatleast=<N>	1-minute load average per CPU	fires at or above N. 1.0 = one runnable process per core. The quantitative form of minload=; see why the busy-worker ratio is often the wrong signal

Five of those consult module state rather than request text — a stat, a classifier, a cookie's authenticated contents, a load sampler read two ways. That is the family's actual shape: match a request on anything known about it.

Globs take * wildcards and a trailing $ anchor. Named-cookie predicates (cookie=<n>, cookie=<n>~<substr>, bs-cookie=<state>) stay on the cookie conditions above — that vocabulary does not compress into one key.

What a matching rule can do. Four outcomes, set by the action keys:

Intent	Keys	Effect
Block	BotShieldRespond 403 (family default)	Refused from the policy walk. No scoring, no cookie mint, no render.
Challenge	BotShieldChallenge noninteractive	Invisible auto-submitting check. interactive for the visible one.
Captcha	BotShieldChallenge captcha	The configured provider's widget.
Move a named score	BotShieldScore <name> +<n>	Moves a per-request accumulator and lets the walk continue.
Decide nothing	BotShieldNoChallenge	Records the match, skips scoring, hands the request to the real handler.

BotShieldChallenge accepts noninteractive, interactive and captcha. It composes by MAX with the score-derived tier and any flag tier floor — it raises the floor, it never downgrades.

It sets that floor on this request only. That is the difference from BotShieldFlagIP plus a rule reading flagged=, which writes per-address state with a window: a client that solves the challenge would keep being re-challenged for the life of a flag it cannot clear. Flag when you want the reputation to persist; challenge when you want to gate the request in front of you.

BotShieldScore — combine weak signals within one request

<BotShieldRule bot-ish>
    BotShieldUserAgent   @scraper
    BotShieldScore       suspicion +10
</BotShieldRule>

<BotShieldRule newcomer>
    BotShieldFirstSight  yes
    BotShieldSolved      no
    BotShieldScore       suspicion +5
</BotShieldRule>

<BotShieldRule act-on-suspicion>
    BotShieldScoreAtLeast  suspicion 15
    BotShieldChallenge     noninteractive
</BotShieldRule>

BotShieldScore <name> +N|-N|=N moves an accumulator; BotShieldScoreAtLeast <name> <N> matches on one. Two spellings so a line is unambiguously an action or a condition.

For the case where nothing is individually damning but the combination is: three of these five and I want a challenge. Writing that as rules alone costs ten rules; this costs four lines.

It lives for one request and then it is gone. That is the whole design. Reputation that outlives a request is a flag — BotShieldFlagIP / BotShieldFlagSession — which is named, discrete, and visible in the decision log. A number that cannot outlive the request cannot quietly bill a client: a request refused at a rate ceiling for someone else's traffic spike leaves nothing behind.

It still moves under BotShieldEnabled LogOnly, and not under mode=observe. The two suppress different things. mode=observe is one rule its author asked to contribute nothing, so it contributes nothing. LogOnly is do not act, and tell me what you would have done — and a decision computed without its inputs is not the decision that would have been made, it is a quieter one reported as if it were the same. A scope in LogOnly whose scores were suppressed would under-report every challenge it was about to start raising, which is the one thing you turned it on to see.

The line is per-request evidence against effects that outlive the request. A named score is the former and still moves; a flagip= write is the latter and stays suppressed.

Order decides what a reader sees. A rule reads what rules above it have accumulated, so BotShieldScoreAtLeast placed above its contributors sees 0. That is not a limitation to work around, it is what makes the value well defined at every point in the ladder — and it is why a score can be read as a condition here at all.

Names scope the coupling. With a single anonymous total every contributor affects every reader, which is what made weights hard to tune: change one and every threshold moves. With names, grep suspicion finds every line that moves it and every line that reads it, and an unrelated accumulator cannot perturb this one.

A rule that only moves a score does not respond — it passes and the walk continues, the same way BotShieldChallenge implies a pass. Without that a scoring rule would refuse with the family default of 403 and everything below it would be unreachable.

Accumulators clamp at ±10000, so a long ladder cannot run away and a reader can bound the value without adding up the file.

= assigns rather than accumulates. It is the sharp one: a later = discards everything earlier rules contributed, which is occasionally what you want and easy to write by accident.

BotShieldChallengeAtLeast — act on an accumulator, after policy

Directive	Syntax	Scope
BotShieldChallengeAtLeast	<name> <n> <tier>	server / vhost / directory

Floors the tier at <tier> when the named accumulator has reached <n>. Rows MAX against each other and against flag and rule tier floors, so several may match and the highest wins; order is irrelevant.

Rows accumulate through nested scopes: a scope adds to what it inherited rather than replacing it. BotShieldChallengeAtLeast none, alone on a line, switches the whole thing off for that scope — no score-driven challenge is raised there, whatever rows the scope declares or inherits, and wherever the line appears relative to them.

Position-independence is the point rather than an implementation detail. A reset that only cleared what preceded it would be unusable to anything that cannot choose where its configuration lands — a generated vhost, an included fragment, a test harness inserting at a fixed anchor. The cost is that "switch off, then add my own" cannot be said in one scope; use a nested one, which is what scopes are for.

This is the off switch a list needs and a single-valued threshold never did: overriding the single noninteractive cut-point silenced it, and there is no equivalent for a row you inherited and cannot see.

<BotShieldRule sig-scraper>
    BotShieldUserAgent  @scraper
    BotShieldScore      botsignals +10
</BotShieldRule>
<BotShieldRule sig-newcomer>
    BotShieldFirstSight yes
    BotShieldSolved     no
    BotShieldScore      botsignals +5
</BotShieldRule>

BotShieldChallengeAtLeast  botsignals  20  noninteractive
BotShieldChallengeAtLeast  botsignals  50  interactive

Why this is a directive and not a rule condition. BotShieldScore and BotShieldScoreAtLeast both live in the rule ladder, which runs before robots.txt and rate limiting. Accumulating there is harmless — nothing between those stages reads an accumulator. Deciding there is not: a request that is both over the threshold and over a rate ceiling would be challenged instead of refused, because the ladder short-circuits before the limiter ever runs.

That is the wrong way round — refusing a bot outright is cheaper than serving it a proof-of-work challenge you were going to reject anyway — and it is not hypothetical: writing the ladder that way broke seven rate-limit tests and a robots test.

So this directive is evaluated with the score-to-tier decision, after all of policy, which is where the tier has always been chosen. Use BotShieldScoreAtLeast inside a rule when you want the accumulator to gate that rule's match; use BotShieldChallengeAtLeast when you want it to choose a challenge tier.

flagged= — does this client already carry a flag

<BotShieldRule escalate>
    BotShieldFlagged   scanner_probe
    BotShieldPath      /admin/*
    BotShieldChallenge captcha
</BotShieldRule>

Both subjects. A flag matches whether it was written to the address with BotShieldFlagIP or to the session with BotShieldFlagSession. That matters most for the three app_* credits, which are refused on an address and so exist nowhere but a cookie — a rule could not see them at all until this read both.

A session flag counts only from a cookie whose authentication tag verifies. That is the same gate the tier decision uses, and the reason for it is symmetrical: a client that could hand itself a flag could also hand itself a credit.

The address half is re-probed per rule, so a rule sees flags written by rules above it in the same walk. The session half is what the client presented: a BotShieldFlagSession written earlier in this same walk is staged for the response and is not visible until the next request, because the cookie carrying it has not been resealed yet.

Flag names are the registered set: honeypot_hit, scanner_probe, fake_bot, pow_fail_streak, app_verified_human, app_verified_session, app_trust_signal, blocked, rate_abuse, robots_ignored. Naming one that is not on the list prints the list, so the authority is the module rather than this paragraph.

rate_abuse and robots_ignored are written by the module, not by your rules. Every other flag needs a BotShieldFlagIP somewhere; these two are set wherever a request is refused for exceeding a rate budget or for fetching a path robots.txt disallows. They exist because those refusals return from inside the policy walk and end the request there -- nothing downstream records them, and the rule walk runs before all of them, so without a flag a client refused a hundred times running is still a stranger to every rule on request 101.

They carry no effect of their own. Nothing is seeded for them, so a write changes no decision until you ask for one:

<BotShieldRule tighten-on-repeat-offenders>
    BotShieldFlagged   rate_abuse
    BotShieldChallenge noninteractive
</BotShieldRule>

The lifetime of a rate_abuse flag is the budget window the client overspent, clamped to between a minute and a day -- a per-minute limit remembers for a minute, an hourly one for an hour, so the flag tracks the limit that produced it without a second number to keep in sync. Where a BotShieldEscalate fired, its ttl= is used instead, because the operator has already said how long an escalated client stays escalated. robots_ignored has no window to derive from and lasts an hour.

Neither is written under mode=observe or BotShieldMode observe (in <BotShieldRobots>). A flag written there would follow the client into later requests and change what matches, which is enforcement by another route and exactly what observe defers.

The read is live. A rule sees flags written by rules above it in the same walk, the same contract BotShieldScoreAtLeast has for accumulators — a rule reads what rules above it have done. So a trap and its response can be one request rather than two:

<BotShieldRule trap>
    BotShieldPath      /.env
    BotShieldFlagIP    scanner_probe
    BotShieldScore     probes +10
</BotShieldRule>

<BotShieldRule act>
    BotShieldFlagged   scanner_probe
    BotShieldRespond   404
</BotShieldRule>

The BotShieldScore line in the trap is load-bearing and easy to miss. A rule whose only action is a flag write ends the walk, so without something that keeps it going — a score movement or a tier — nothing below it runs and the reader never fires. That is ordinary rule behaviour rather than anything flagged= introduced, but it is the difference between the pattern above working and silently not.

The cost of a live read is that rule order is load-bearing. Put the reader above the writer and it fires from the next request instead of this one. That is the same trade BotShieldScoreAtLeast makes, and it is why both read the same way rather than each choosing.

A rule may not match a flag and write the same flag. It is a config error:

# configtest: skip -- this block is the example of what is refused;
# httpd rejecting it is the documented behaviour.
# refused
<BotShieldRule selffeed>
    BotShieldFlagged   scanner_probe
    BotShieldFlagIP    scanner_probe
</BotShieldRule>

Such a rule refreshes the flag's expiry on every request it matches, so the address never ages out of it — and expiry is the only recovery for a client that cannot solve a challenge. (A client that can is covered by the tier decision, which does not re-ask a question the cookie already answers.) The match earns nothing there either: if the rule's other conditions justify the write, they justify it whether or not the flag is already set. Write the flag from the rule that detects the behaviour.

Matching one flag and writing a different one is allowed, and is the escalation shape: scanner_probe on a sensitive path becomes honeypot_hit.

This check is per-rule. Rule A writing a flag while rule B matches it and writes it again is the same hazard spread across two rules, and catching that would mean reading the config as a graph — so the check is honestly partial rather than pretending otherwise.

There is no !flagged=. An address not carrying a flag is the ordinary case, and a rule conditioned on it fires on nearly every request; say what you mean with the other conditions. Negation is accepted only on cookie= and env=, where the complement of "this named thing is present" has no name.

cookie= and env= — the two open-set conditions

Every other condition here is an enumeration whose complement has a name: solved=no, crawler=no, cookies=any. These two are not — an arbitrary cookie or environment variable is an open set, and "not present" has no other spelling. So they are the only conditions that take a negation, written as a leading !:

Written	Matches
BotShieldCookie NAME	the cookie is present
BotShieldCookie !NAME	it is absent
BotShieldCookie NAME=VAL	present and equal
BotShieldCookie NAME!VAL	present and not equal
BotShieldCookie NAME~SUB	present and contains

!NAME and NAME!VAL are different questions and the parser keeps them apart: a client sending no cookie at all has not sent a different value. !NAME=VAL is refused rather than guessed at.

The first operator after the name wins and the rest is the value verbatim, so NAME=a~b is an equality test against the literal a~b.

BotShieldEnv takes the same forms minus ~:

BotShieldEnv   BS_LEVEL          # set
BotShieldEnv  !BS_LEVEL          # not set
BotShieldEnv   BS_LEVEL=high     # set to exactly this

This is also the answer to "why is there no regex". The module has none anywhere — paths are globs, UA is substring or @cohort — and adding one would mean running an operator-supplied pattern against attacker-controlled input on every request, which is a denial-of-service surface pointed the wrong way. Apache already ships regex engines that are tuned and audited, and env= is how they compose:

SetEnvIfExpr "%{HTTP_USER_AGENT} =~ /bot|crawl|spider/i" BS_UA_SUSPECT=1

<BotShieldRule ua-regex>
    BotShieldEnv       BS_UA_SUSPECT
    BotShieldPath      /search
    BotShieldSolved    no
    BotShieldChallenge noninteractive
</BotShieldRule>

That composition is the reason these belong in the rule rather than in a family of their own. The family they replaced matched a cookie and nothing else; a rule ANDs it with the path, the UA, the load state and whether the client has already solved.

The module's own __Host-bs_session is refused here — use BotShieldBSCookie, which distinguishes verified from missing from invalid, a distinction a presence test would flatten.

@selectors on ua=

Four of them name a classification this module makes rather than a bot the directory knows:

Selector	Matches
@bot	verified, known and unknown bots — the same three the dashboard's Bots tab counts. Not fake-bot: a client lying about being a crawler must not inherit an exemption written for real ones
@verified-bot	the narrow half of @bot: the UA matched a crawler pattern and the address checked out against that crawler's published ranges
@fake-bot	a UA claiming a crawler whose address failed the cross-check
@scraper	the UA carries a known HTTP-library token — curl, wget, python-requests, Go-http-client, okhttp, scrapy and similar

@bot and @verified-bot answer different questions and the width is the point. @bot is for acting on bots — rate limits, robots policy, anything where a bot-shaped client should be treated as one whether or not it has proven who it is. @verified-bot is for exempting a crawler you have actually confirmed, and admitting unknownbot to that would hand the exemption to any client with a bot-shaped UA.

@verified-bot is also the one selector that cannot be approximated by crawler=yes. That predicate reads the UA's claim and requires a directory entry, so a crawler you added yourself with BotShieldAllowBot answers crawler=no however thoroughly its address was verified.

Anything else after @ is read as a botgroup name from the bot directory (search, ai-input, ai-train, monitor).

@scraper is the scraperua signal as a condition, and it reads the same token list the heuristic scores — one list, so a rule and a score cannot disagree about what a scraper is. It is a class rather than a botgroup because curl and python-requests are not crawlers with names; they are clients that did not claim one.

BotShieldAcceptLanguage — absent, or present

"" matches a request with no Accept-Language header or an empty one, * matches a request that carries it. It is spelled like BotShieldUserAgent "" because it asks the same kind of question, and absence is not a substring.

Substrings are refused at config time. This is the missingal signal as a condition, not a general header matcher — that is a larger surface with its own escaping and case rules, and it wants deciding on its own merits rather than arriving by way of a value slipping through.

BotShieldFirstSight — has this address been seen before?

yes matches an address the Bloom filter has no record of, no matches one it does. A request with no usable client address matches neither, the same call exists= makes when it cannot stat.

Paired with solved= it is exactly what the two address heuristics measure: firstsight=yes solved=no is firstsightip, and firstsight=no solved=no is droppedcookie. They are two halves of one signal split on Bloom membership, and this is the half the rule family was missing.

What it adds over the heuristic is scope. firstsightip applies to every path in a scope or to none, at one weight; the recommendation to use it "on a login or registration path" was not writable. Now it is:

<BotShieldRule newcomer-gate>
    BotShieldPath        /login
    BotShieldPath        /register
    BotShieldFirstSight  yes
    BotShieldSolved      no
    BotShieldChallenge   noninteractive
</BotShieldRule>

The read is resolved once per request and happens before that request's write, so two rules naming this key cannot disagree and the heuristics read the same answer.

What the filter actually holds is worth knowing before you use this. An address is recorded only on requests that reach the cookie mint, so a request some rule refused does not register the client. That is deliberate rather than an oversight: droppedcookie means "known address arriving without a usable cookie", and the suspicion in it is that the client was given a cookie and is not presenting it. A refused request never reached a mint, so that client has nothing to present, and recording it would make droppedcookie a penalty for having been blocked once.

So this filter answers was this address given a chance to hold a cookie, not has this address been seen. The practical consequence: a client that only ever hits refusing rules stays first-sight, and a rule combining firstsight=yes with a refusal will match that client every time rather than once.

BotShieldNoChallenge

Takes no argument. The rule records its match, skips scoring, and declines out of the handler so the real site serves the request.

It is named for what it waives. The request still meets rate limiting and robots.txt, and the rule's own flag writes still happen — so this is not an exemption from BotShield, only from the challenge decision. An earlier spelling, pass, was retired for inviting the opposite reading, and Exempt would have claimed more still.

Skipping scoring is the part to be deliberate about: it disables challenges the defaults would otherwise have raised on those requests. It is a logging-and-flagging form, not a no-op, and grep BotShieldNoChallenge over a config lists every place protection is switched off.

# a real file under /administrator/ is served normally; the rest of the
# ladder below only ever sees paths that do not exist.
<BotShieldRule admin-exists>
    BotShieldPath         /administrator/*
    BotShieldExists       yes
    BotShieldNoChallenge
</BotShieldRule>

Removed: BotShieldLog

BotShieldLogAs <tag> is the name now. The old one was removed 2026-09-06 with the other four retired spellings and fails config parse naming its replacement -- it does not parse with a warning, and this section said it did for a day longer than that was true.

It never caused logging. The decision line is emitted whether or not a rule sets a tag, and the tag is embedded on that same line rather than producing a second entry — so BotShieldLog read as the thing that made the record happen, and as something you could delete to stop one. Neither is true. As says the value is a name for a line that was always going to exist.

Distinct from BotShieldAccessLog, which is a genuine on/off switch for the Apache access-log line and is unaffected by this rename.

Removed: BotShieldTier, and BotShieldRespond nochallenge

Both were removed 2026-09-06 and now fail config parse, naming their replacement. This is the only deployment running the module, so there was nobody to hold a deprecation window open for.

BotShieldTier required BotShieldRespond nochallenge beside it, because a concrete status short-circuits before any tier is chosen. That companion carried no meaning of its own — you wrote it to be allowed to write the line you wanted. BotShieldChallenge sets both, so it goes away. It also names an act rather than a taxonomy: "tier" is what the levels are called internally, "challenge" is what the rule does.

A bare BotShieldRespond nochallenge — with no tier and no penalty — was how you spelled "decide nothing", by omission. That is now BotShieldNoChallenge, said outright.

Old	New
BotShieldRespond nochallenge + BotShieldTier noninteractive	BotShieldChallenge noninteractive
BotShieldRespond nochallenge + BotShieldScore botsignals +20	BotShieldScore botsignals +20
BotShieldRespond nochallenge (bare)	BotShieldNoChallenge

# cookieless crawler walking a login redirect chain: path AND query AND
# cookie-state, one cheap 403 from the policy walk, tagged for fail2ban
# and kept out of the access log
<BotShieldRule login-trap>
    BotShieldPath         /login*
    BotShieldQuery        *return=*
    BotShieldCookies      none
    BotShieldRespond      403
    BotShieldLogAs        login-trap
    BotShieldAccessLog    off
</BotShieldRule>

# no path condition at all — any URL carrying ?debug=1
<BotShieldRule debugparam>
    BotShieldQuery        *debug=1*
    BotShieldScore  botsignals +20
</BotShieldRule>

# no User-Agent at all. Absence is not a substring, so this is the one
# UA form the pattern match cannot express. Note ua="" is a restriction
# and ua=* is not: "*" (or omitting the key) means "any", which is why a
# rule carrying only ua=* is rejected as having no condition.
<BotShieldRule no-ua>
    BotShieldUserAgent    ""
    BotShieldChallenge    noninteractive
    BotShieldLogAs        no-ua
</BotShieldRule>

Because it fires from the policy walk it short-circuits before scoring, so a respond=4xx rule never renders a challenge and never reaches PHP.

Remembering a client is opt-in. A rule fires, responds, and forgets — unless you ask it to remember. Two directives, two subjects, and neither happens unless the rule says so.

BotShieldFlagIP <flag> marks the address. Everything sharing that address is marked with it: the point behind a hosting range, the problem behind a residential NAT.

BotShieldFlagSession <flag> marks the cookie session. The mark travels inside the cookie the client keeps handing back, so it catches the one browser that tripped the rule and nobody else sharing its address. A client that throws cookies away escapes it — and meets the full challenge gate regardless, because it can never hold a solve either, so it costs nothing that was not already being paid.

# the address: right when the source itself is what you distrust,
# wrong when it is a NAT with real users behind it.
<BotShieldRule env-probe>
    BotShieldPath         /.env
    BotShieldRespond      404
    BotShieldFlagIP       scanner_probe
    BotShieldLogAs          env-probe
</BotShieldRule>

# the browser: the probe gets its 404 and a cookie carrying the mark,
# and the neighbours are untouched.
<BotShieldRule wp-probe>
    BotShieldPath         /wp-admin/*
    BotShieldRespond      404
    BotShieldFlagSession  scanner_probe
    BotShieldLogAs          wp-probe
</BotShieldRule>

A flag does nothing on its own. What it means is declared once, by a rule matching flagged=, and nothing is built in. A rule that sets a flag no trigger defines is inert.

Which flags may go where

Three describe a session and are refused on an address:

Flag	Subjects
honeypot_hit, scanner_probe, fake_bot, pow_fail_streak	address or session
rate_abuse, robots_ignored	address (written by the module)
app_verified_human, app_verified_session, app_trust_signal	session only

Those three are credits. Suspicion shared across an address costs strangers a challenge, which is the trade this module already makes. A credit shared across an address hands strangers an exemption someone else earned — one person logging in would discount everyone behind that NAT, renewed by whatever traffic keeps the address flagged. Refused at config time, because at runtime it looks like a rule that works.

How long an address stays flagged

BotShieldForgetIPAfter <seconds> at server scope. Default 3600, range 1..2592000.

It is not per rule, and not per flag. One address slot holds one expiry shared by every flag on it, so two rules asking for 3600 and 86400 both get 86400 — a per-rule duration reads like it works and cannot.

It slides: the clock runs from the address's last flagging, not its first. So an address that keeps tripping rules stays flagged, and one that stops is forgotten a window later. A mark set early is therefore not cut short by a later one; it is carried while there is still reason to distrust the address.

Sessions take no duration. A session mark lives exactly as long as the cookie does, which is what a session is.

Flags are advisory, not enforcement. The table holds 50,000 entries and evicts live ones under pressure, so a flag can lapse before its window is up. It is a strong hint, not a guarantee.

Adding, removing, replacing

+name adds, -name removes, =name makes the named set the whole set. + is the default and may be omitted, so an unprefixed list means what it always did. Mixing = with + or - is a config error — =a,+b has no reading that is not a guess. The same grammar and the same rule as BotShieldClassify.

- and = are only allowed on BotShieldFeedback. A rule matches on request properties the client controls, so clearing there would let anyone shed their own record by fetching the URL that matches. Feedback fires on a header your application signs: the application asserts it, the visitor cannot.

<BotShieldFeedback login-success>
    BotShieldEvent        login-success
    BotShieldFlagSession  +app_verified_human,-scanner_probe
</BotShieldFeedback>

Feedback marks the cookie the response is already carrying rather than adding a second one. If the response carries no cookie at all, there is no session to mark and the module logs that the flag was not applied rather than creating one.

Removed: BotShieldFlag and BotShieldTTL

Both were removed 2026-09-06 and now fail config parse.

BotShieldFlag named no subject, and the subject is the whole question: an address is shared and a cookie is not. Write BotShieldFlagIP or BotShieldFlagSession.

BotShieldTTL set a per-rule duration that was never honoured — one address slot holds a single expiry shared by every flag on it, extended to whichever rule wrote last, so the number was read, stored and partly ignored. The window is BotShieldForgetIPAfter, at server scope, said once where it is true.

Retiring BotShieldFlag exposed a bug it had been hiding: the app feedback filter read only the field that spelling set, so BotShieldFlagIP on a <BotShieldFeedback> block passed config parse and then did nothing. Both subjects work there now.

This family used to flag by default, scanner_probe for 3600 s, inherited from BotShieldPathTrigger where the target was a handful of scanners probing /.env. It was the wrong default for high-cardinality traffic: at roughly one request per address the flag is never read again and it churns the 50,000-slot table. And a scanner_probe flag is not inert: wherever a rule gives it a BotShieldChallenge — as the seeded slate did before default rules were switched off — that floor overrides parked score thresholds, so a rule written to block quietly began rendering interstitials to whoever shared that address next, with nothing in the config saying so.

Migrating: a rule written before 2026-09-05 that relied on the default now flags nothing; add BotShieldFlagIP scanner_probe to keep it. BotShieldTTL 0, which used to be how you switched the default off, is what the default already is and can be deleted.

Removed: BotShieldRequestTrigger

BotShieldRequestTrigger was the old spelling of this directive. The family stopped being about requests-versus-something-else once it grew ua=, ipspec=, query=, cookies=, exists=, solved= and minload=. What it actually does is match a request on any combination of its properties and act once, which is what a rule is.

It was deprecated 2026-09-05 and removed 2026-09-06 — one day rather than the nine BotShieldPathTrigger got, because a census found no config anywhere using it. A block with the old tag now fails config parse, like any unknown directive. Rename the block and its closing tag:

# configtest: skip -- the "before" half names a removed directive and
# is here to be recognised, not run.
# before
<BotShieldRequestTrigger blocked>
    BotShieldPath         /wp-admin/*
    BotShieldRespond      403
</BotShieldRequestTrigger>

# after
<BotShieldRule blocked>
    BotShieldPath         /wp-admin/*
    BotShieldRespond      403
</BotShieldRule>

Nothing else changes: same conditions, same actions, same parser, same resulting rule.

The decision log's reason prefix moved with it. A rule that fires now logs rule:<name> where it used to log requesttrigger:<name> — the old prefix was a separate literal and would otherwise have been the only surviving trace of a directive nobody can write.

Removed: BotShieldStatus

BotShieldStatus is the old spelling of BotShieldRespond. Apache already spends the word "status" on mod_status and server-status, and this module ships a dashboard and a metrics endpoint of its own, so in a BotShield config the old name read as a monitoring surface rather than as the response a rule produces. Nothing in Apache names a response code Status either: Redirect and ErrorDocument take one as an argument, and mod_rewrite spells it [R=404].

The old name was removed 2026-09-06 and now fails config parse. BotShieldEscalate had its own status= key, removed with it -- one concept should not wear two names in the same file, and it was wearing two in two.

BotShieldLog went the same day, for a reason of its own: it read as the thing that produces the log entry, and removing it as a way to stop one. It does neither. The line was going to be emitted anyway; BotShieldLogAs only labels it, and the "As" says the value is a name.

<BotShieldRule blocked>
    BotShieldPath         /wp-admin/*
    BotShieldRespond      404
</BotShieldRule>

Values are unchanged: an HTTP code 100..599, or nochallenge.

Migrating from BotShieldPathTrigger

BotShieldPathTrigger was renamed on 2026-08-01 and its path glob moved from a positional argument to the path= key. The old name had stopped being accurate when the family gained ua=/ipspec=; query=, cookies= and exists= made "path" one dimension out of six. Making path a key is also what allows a rule with no path condition at all.

The old name was removed on 2026-08-10 and now fails with Apache's Invalid command at config time — loudly, at startup, not silently at runtime:

# configtest: skip -- the "before" half is a removed directive and is
# meant to be refused; that refusal is the point of the example.
# before
BotShieldPathTrigger blocked "/wp-admin/*" status=403
# after
<BotShieldRule blocked>
    BotShieldPath         /wp-admin/*
    BotShieldRespond      403
</BotShieldRule>

A note on quoting: values are unquoted by the module, so path="/login*" and path=/login* behave identically. Apache's TAKE_ARGV tokenizer only strips quotes that begin a token, so without this a quoted key="value" would retain its quotes and silently never match.

Removed: BotShieldTrigger

The per-Apache-scope family, removed 2026-09-06. Its predicate was the container match and it carried no conditions of its own, which is a rule whose condition Apache has already evaluated — so it could say nothing a rule could not, once a rule could be written in a container.

Write <BotShieldRule name> in the same container. The block name is the only addition; the action keys are unchanged.

# configtest: skip -- the "before" half names a removed directive.
# before
<Location "/admin/.env">
    <BotShieldTrigger>
        BotShieldFlagIP   honeypot_hit
    </BotShieldTrigger>
</Location>

# after
<Location "/admin/.env">
    <BotShieldRule admin-trap>
        BotShieldFlagIP   honeypot_hit
    </BotShieldRule>
</Location>

Two behaviours changed with it. Several BotShieldTrigger blocks in one scope all fired; rules are a ladder and stop at the rung that matches. And reset, which dropped triggers inherited from outer scopes, is gone — a nested scope opts out by writing a rule that matches and passes, which works because the inner scope is walked first. See policy.

Removed: BotShieldFlagTrigger

The family that said what a flag was worth, removed 2026-09-07. It mapped a flag bit to one of three actions at the tier decision, and a rule matching flagged= does all three from the policy walk:

Was	Now
action=score accumulator=A add=N	BotShieldScore A +N
action=tier_floor min=<tier>	BotShieldChallenge <tier>
action=block status=N	BotShieldRespond N

# configtest: skip -- the "before" half names a removed directive.
# before
<BotShieldFlagTrigger honeypot_hit>
    BotShieldAction       score
    BotShieldAccumulator  botsignals
    BotShieldAdd          60
</BotShieldFlagTrigger>

# after
<BotShieldRule flag-honeypot-hit>
    BotShieldFlagged      honeypot_hit
    BotShieldScore        botsignals +60
</BotShieldRule>

Three things to know when migrating.

Declare them first. A rule carrying only BotShieldScore continues the walk, so several in a row all contribute — but a rule above them that answers with a status ends the walk, and the flag rules below it never run. The family applied at the tier decision and so had no such ordering; rules do.

reset is gone. It cleared earlier declarations for a flag, including compiled-in defaults. Nothing is seeded any more, and rules settle by declaration order, so what reset expressed is now "declare the one you want".

A block is a rule that answers. action=block refused a non-refusal status, since a block that answers 200 is a contradiction. BotShieldRespond makes no such claim and takes any code 100..599, so that check went with the verb.

Request signals

There is no directive family for these. Each was a named heuristic an operator bound a score action to; each is a rule condition now, and the weight is a BotShieldScore line inside the rule.

Signal	Fires when	Written as	Suggested
missing UA	User-Agent absent or empty	BotShieldUserAgent ""	+40
missing AL	Accept-Language absent	BotShieldAcceptLanguage ""	+5
scraper UA	UA contains a known HTTP-library token	BotShieldUserAgent @scraper	+10
first sight	Bloom-filter miss — genuinely new address	BotShieldFirstSight yes	+5
dropped cookie	Bloom-known address arriving with no usable cookie	BotShieldFirstSight no	+25

<BotShieldRule sig-scraperua>
    BotShieldUserAgent  @scraper
    BotShieldScore      botsignals +10
</BotShieldRule>

The family bought one thing a rule does not: a scope could re-weight a signal without restating its condition. What it cost was a second vocabulary for conditions the rule language already had, five predicates compiled in where an operator could not read them, and a reset verb per name to undo defaults that should not have been there. See tests/setup/botshield-dev.conf for the five written out.

Nothing is seeded. A signal with no rule contributes nothing.

scraperua is deliberately low. robots.txt tells undeclared clients they may fetch anything outside the Disallow list at the published Crawl-delay; a weight of 50 put an unrenderable checkbox in front of curl, wget and python-requests instead — the module enforcing a policy the site never published. At 10 it composes with other signals rather than deciding on its own, and volume abuse is caught by the rate limit, which is what the published policy actually promises. missingal is 5 for the same reason: almost nothing scripted sends Accept-Language.

firstsightip and droppedcookie are the two halves of one signal — both fire unless the request carries a cookie that proves a challenge was solved, differing on whether the IP is already in the Bloom filter.

A merely valid cookie is not enough. Under always-mint every client receives a signature-valid cookie on its first request, so waiving these two on validity alone would let a bot mint one, keep it, and permanently suppress a penalty it never earned. The waiver requires solve evidence (passes_silent, passes_form, or passes_captcha) in the authenticated rep block — the same evidence the safeguard-clear path requires. A real browser pays for this exactly once: it arrives with no proof, scores droppedcookie, clears it in one auto-submitted round trip, and every later request carries proof.

A dropped cookie is ambiguous by design — private-browsing resets and manual cookie clears look the same as evasion — so its weight is deliberately mild, and first sight milder still. Setting either at or above your lowest BotShieldChallengeAtLeast row makes that scope challenge any request with no session context. That is usually what you want on a login or registration path, where the check is invisible and a real browser clears it in one round trip; it is usually not what you want site-wide.

Safeguard

One container, <BotShieldSafeguard>, once per server scope. It takes no argument, and the settings inside it drop the feature name they used to repeat.

<BotShieldSafeguard>
    BotShieldThreshold    5
    BotShieldWindow       600
    BotShieldTTL          900
    # Optional. When unset, the redirect points at
    # /botshield/safeguard-info (the module's built-in explainer).
    BotShieldRedirectURL  /help/auto-check-failed
</BotShieldSafeguard>

Inside the container	Syntax	Default
BotShieldEnabled	On|Off	On
BotShieldThreshold	N (1..1000)	5
BotShieldWindow	N sec (1..86400)	600
BotShieldTTL	N sec (1..604800)	900
BotShieldRedirectURL	/path, same-origin	unset (uses built-in explainer)

The block is optional: safeguard runs on the defaults above without one. Write it to tune those numbers, or to turn the feature off:

<BotShieldSafeguard>
    BotShieldEnabled Off
</BotShieldSafeguard>

BotShieldSafeguardCapacity sizes the SHM table and stays outside the block. It is module-global — only the main server's value is read at post_config — so unlike the settings above it cannot be set per-vhost, and a block written inside a <VirtualHost> could not have carried it. See SHM sizing.

Challenge-loop suppression, on by default — only an explicit BotShieldEnabled Off inside the block disables it. A client that cannot solve the challenge (JS disabled, a privacy extension, an old browser) would otherwise be re-challenged forever with nothing in the logs shouting about it, and that client is indistinguishable from a non-JS crawler. The redirect resolves that without having to tell them apart: it is useful to a human and useless to a crawler. The tripped client is not admitted — it lands on an explainer, never on protected content, and its flagged-IP entry survives.

When a client has been issued the threshold number of challenges within the window without ever returning a verified cookie, the next request gets a 302 redirect to break the loop.

BotShieldRedirectURL lets the operator point the redirect at their own page (a status page, a help article, a login flow). When unset, the module redirects to its built-in explainer at <BotShieldEndpointPrefix>/safeguard-info. The original URI is appended as ?return=<urlencoded path> regardless of which target is chosen, so the user can resume their journey once the underlying problem is fixed. The return parameter is validated for same-origin shape (must start with a single /, no scheme, no double-slash) to prevent open-redirect abuse.

The built-in explainer page describes common reasons the auto-check failed (JavaScript disabled, privacy extension, browser version) and offers a Continue link back to the original URL. It is auto-routed by the module — no <Location> carve-out needed.

See policy and site model for the behavior arc.

Load-aware throttling

Sampling and hysteresis:

Directive	Syntax	Default	Scope
BotShieldLoadStateFile	/path	unset	server only
BotShieldLoadRefreshInterval	N (sec)	1	server only

The six warm/hot thresholds were directives until 2026-09-07 and are now fixed constants in src/shm.h: busy-worker ratio 65/85 percent, mean latency 250/1000 ms, load average 1.00/1.50 per CPU. Nothing had ever configured them, and the state they drive stopped being policy when the shed ladder moved to BotShieldLatencyAtLeast — a rule reads the raw number, not the state. What is left is a gauge, and a gauge with tunable bands nobody tuned is a knob with no reader. | BotShieldDbStatsFile | /path | /run/botshield/db-load.stats | server only | | BotShieldFpmStatsFile | /path | /run/botshield/fpm-load.stats | server only |

BotShieldLoadStateFile points at an external single-word state file (managed by an out-of-band collector) that overrides the scoreboard sample. Useful when load decisions should key on a metric Apache itself doesn't see (queue depth, downstream saturation, etc.).

Why the busy-worker ratio is often the wrong signal

The busy-worker warm/hot thresholds are a percentage of MaxRequestWorkers, which is only meaningful if that setting reflects what the machine can actually serve. It frequently does not. On a host running MaxRequestWorkers 1024 against 6 cores, four separate outages ran at 25-30 busy workers — the site returning 500s and taking half a minute per request — which is 2-3% utilisation. No threshold on that ratio can distinguish those outages from an idle server.

That is not true of every outage, and the ratio is not the only way to read the scoreboard. Replayed from the access log, the outages of 2026-08-20 and 2026-08-27 ran near 950 requests in flight and the floods of 2026-08-08 and 2026-08-16 near 980, against an ordinary-day peak of 119-181 that week. What misleads is dividing by a ceiling that does not describe the machine; the count itself separated those days cleanly. BotShieldBusyWorkersAtLeast reads the count.

The latency thresholds exist for that case. They compare the mean request latency, measured as a delta between watchdog ticks, against a fixed duration — and BotShieldLatencyAtLeast lets a rule compare against one you choose. On the host above the same outages moved this number from ~31ms to 29,000-36,000ms — roughly a thousandfold, on the same data the worker ratio read as flat.

It is derived from Apache's own per-worker counters, the same ones mod_status sums for Total Duration and Total Accesses, read straight out of the scoreboard the watchdog already walks. No extra sampling cost and no external process. Two consequences worth knowing:

• It requires ExtendedStatus On. Without it Apache never maintains those counters and the metric reports unavailable rather than zero — zero would mean "answering instantly", which is the opposite of what a missing measurement means.
• It averages over all requests, static files included, so a flood of cheap static hits dilutes it. It answers "is the server slow right now", not "is this endpoint slow".

BotShieldDbStatsFile reads key=value telemetry from an external database monitor for the dashboard's graph. Database load reaches policy through BotShieldLoadStateFile, not through this — the module never links a database client, because blocking I/O has no place in the watchdog and a database too sick to answer must not be able to stall the code whose job is to shed load because the database is sick.

Reaching a number from a rule

BotShieldLoadAvgAtLeast <N> inside a <BotShieldRule> matches on the per-CPU load average directly, in the same hundredths-per-core unit the warm/hot bands use:

<BotShieldRule shed-scrapers-under-load>
    BotShieldLoadAvgAtLeast  2.0
    BotShieldUserAgent       @ai-train
    BotShieldRespond         503
</BotShieldRule>

That is the same sample the watchdog reads to decide warm and hot and the same one botshield_loadavg_1m_pct reports — no second source of truth, and the gauge on the dashboard is the number the rule compares against. Per CPU, so a threshold means the same thing on a 6-core host and a 64-core one.

Rate-limiting from a rule

BotShieldDelay <seconds> or BotShieldRate <n> <seconds> puts a fixed-window counter on a rule. Once the window is spent the rule applies its action -- 429 unless it says otherwise. Under budget the request spends from the window and the walk carries on to whatever else the rule says and to the rules below.

<BotShieldRule crawl-delay>
    BotShieldUserAgent  @bot
    BotShieldDelay      1.5
</BotShieldRule>

<BotShieldRule search-flood>
    BotShieldPath       /search/
    BotShieldRate       30 60
</BotShieldRule>

The unit is always seconds, and seconds take a fraction: 0.5 is half a second, exactly -- the counter keeps milliseconds. There are no unit words; 10min is refused rather than read as ten seconds. The ceiling is 86400.

The two spellings differ in who shares the window, and that is the whole reason there are two:

	one window per	budget	is
BotShieldDelay <s>	crawler (known-bot slug)	1 request	robots.txt Crawl-delay
BotShieldRate <n> <s>	rule	n requests	a shared budget
BotShieldRate <n> <s> each	crawler (known-bot slug)	n requests	a per-crawler budget above one

Delay is Crawl-delay in the unit robots.txt writes it in, and it means what robots.txt means: each crawler gets a window of its own. Rate is one window for everyone the rule matches, between them. BotShieldUserAgent @bot with BotShieldRate 1 1 gives the entire crawler population one request a second shared, which reads almost like BotShieldDelay 1 and is a very different policy. Until 2026-09-08 this was a separate countper= knob whose default was the shared bucket, and the default was the trap.

A rule has one counter, so Delay and Rate on the same rule is refused rather than last-one-wins.

Delay costs a counter slot per entry in the bot directory, which is large against the pool, so post_config allocates what it can and logs a warning naming the rule when it cannot; slugs that missed out share the rule's own slot. A matching request whose UA is not a known bot has no slug to key on and uses that same slot, so unknown bots are one shared window.

BotShieldDelay 0 is accepted and means no limit, so a robots.txt transcription can keep a literal Crawl-delay: 0. A value that rounds to zero milliseconds without being zero is refused: 0.0001 is someone who meant a limit and would silently get none.

What a rule bought over the retired BotShieldRateLimit is the predicate set. That directive matched on ua= and ipspec= and nothing else -- there was no path in it -- so search-flood above could not be written with it at all. On a rule every condition is available: path, query, cookies, solved=, flagged=, latencyatleast=, and the rest.

Not expressible: a per-crawler budget larger than one ("thirty a minute, each"). Rate shares and Delay is one-at-a-time; a per-crawler rate would be a third spelling, and it has not been needed. Nor is a per-client-address window, which is what most people mean by "rate limit"; it needs a counter table keyed on (address, rule) rather than the fixed per-rule slot, and that table is separate work.

Reaching request latency from a rule

BotShieldLatencyAtLeast <ms> matches on Apache's mean request latency, in the same milliseconds the warm/hot bands use:

<BotShieldRule shed-bots-when-slow>
    BotShieldLatencyAtLeast  1000
    BotShieldUserAgent       @bot
    BotShieldRespond         503
</BotShieldRule>

This is the condition the other two cannot express, and the reason is what "busy" means to each of them. A worker blocked on a database socket is in interruptible sleep, which Linux does not count toward the load average, so a server with every worker waiting on a sick database reads as idle to BotShieldLoadAvgAtLeast. Its mean request duration does not read as idle: the blocked worker accumulates duration for the entire time it waits, which is exactly the quantity that makes a page take thirty seconds.

Measured as a delta between watchdog ticks, so it is what the server is doing now rather than an average since restart.

It requires ExtendedStatus On. Apache only maintains the per-worker access and duration counters when that is set; with it off they sit at zero forever. The module reports the metric as unavailable rather than as 0ms, and a BotShieldLatencyAtLeast rule declines while it is unavailable. That direction is deliberate: the alternative is a server that starts shedding traffic at the moment it loses the ability to measure itself.

0 is refused rather than read as "no condition", because a 0ms floor matches every request and a rule saying "shed when latency is at least nothing" is always a mistake.

Why latency is a poor shedding signal

Request duration runs until the last byte is sent, so it includes the time a client spends receiving the response. A slow client downloading a large file holds a request open for minutes while the server does almost nothing, and reads as a slow server. The mean over a sample is also dominated by whichever single request is slowest: on this deployment 91% of the samples that crossed 500ms were one request doing 80% or more of that sample's time, most often one expensive search page.

Latency remains useful on the dashboard. For deciding to turn traffic away, prefer a signal that counts work being done.

Reaching work signals from a rule

Four conditions count something occupied only while work is happening, so none of them moves for a slow download:

Directive	Reads	Unit
BotShieldBusyWorkersAtLeast <n>	Apache worker slots busy at the last watchdog tick	count
BotShieldFpmBusyAtLeast <pct>	PHP-FPM active processes	percent of pm.max_children, 1-100
BotShieldFpmQueueAtLeast <n>	requests waiting for a PHP-FPM worker	count
BotShieldDbRunningAtLeast <n>	database threads running	count

<BotShieldRule shed-bots-when-php-is-full>
    BotShieldFpmBusyAtLeast  80
    BotShieldUserAgent       @bot
    BotShieldRespond         503
</BotShieldRule>

The PHP-FPM and database figures come from the stats files the bundled monitors write (BotShieldFpmStatsFile, BotShieldDbStatsFile), read once per watchdog tick. A sample more than 60 seconds old is treated as no reading, and a rule declines without one, as it does when latency is unavailable: a monitor that stopped writing is neither a calm server nor a loaded one, and shedding on its last word would punish traffic for a stopped service.

The busy-worker count comes from the scoreboard the watchdog already walks and needs no monitor. Set its threshold well below MaxRequestWorkers: once every worker is busy, new connections wait in the kernel's accept queue and never reach the module, so a rung that fires at the ceiling cannot shed anything.

Conditions in one rule are ANDed. To shed on either of two signals, write two rules with the same response; the family is first-match-wins so a request is counted once.

0 is refused on all four, for the reason given for latency, and ! is refused because there is no "below" condition.

A rule carrying any of these, or BotShieldLoadAvgAtLeast or BotShieldLatencyAtLeast, that answers with anything but BotShieldNoChallenge is counted as shedding: see botshield_shed_total and botshield_shed_observed_total in the observability guide, and the Requests shed and Would shed figures on the dashboard.

Choosing between the two

BotShieldLatencyAtLeast is the one that sees a worker blocked on I/O; BotShieldLoadAvgAtLeast is the one that sees CPU saturation. On this deployment the failure mode has been the former, so the shed ladder reads latency and keeps load average in reserve.

There was a third, minload=, matching the three-state normal/warm/hot machine. It was removed on 2026-09-07: the shed ladder was its only consumer, and the state it read disagreed sharply with the load average about what "busy" meant — the machine reached warm 172 times on 2026-09-01 while 98.3% of the shedding it drove happened between 0.07 and 0.24 per CPU. Nobody could account for what moved it, and a condition whose firing cannot be explained is worse than no condition. The measurement stayed: see below.

There is no "below this" spelling and ! is refused. A rule for the quiet case is the one the loaded rule falls through to, which keeps the two thresholds from drifting apart.

After a config reload it reads 0 until the next watchdog tick. A graceful reload builds a fresh SHM header and nothing repopulates the sample until the watchdog runs again, so for up to BotShieldLoadRefreshInterval seconds — one, by default — a load-conditioned rule does not fire. It fails open: requests that would have been shed are served. For a shed rule that is the right direction to fail, and one second of a reload you performed is not a window an attacker can choose. It is documented rather than fixed because the alternative — persisting the value across a reload — means a rule acting on a measurement taken by a configuration that is no longer running.

The three-state machine is not affected: it is recomputed from the scoreboard on the same tick and starts at normal, which is also fail-open.

No rule reads that machine any more. It survives as the load_state gauge and the load_state_changes_total series, which is the right home for a signal that is worth watching and not yet worth acting on. See policy.

Multi-vhost reputation

Directive	Syntax	Default
BotShieldShareScope	<token>	derived from ServerName

Vhosts with the same token share one reputation namespace. See deployment.

Observability endpoint access

Directive	Syntax	Default	Scope
BotShieldDashboardAccess	<addr|cidr>... | all | none	closed	server / vhost
BotShieldMetricsAccess	<addr|cidr>... | all | none	closed	server / vhost

Who may read each observability endpoint. Each is closed until its own directive names someone, and a refused request gets 404, not 403, so a scan cannot tell the endpoint exists.

BotShieldDashboardAccess 127.0.0.1 ::1
BotShieldMetricsAccess   127.0.0.1 ::1

One directive per endpoint, so opening one never opens the other. A Prometheus scraper and an admin browser are rarely the same host:

BotShieldMetricsAccess   10.9.0.5
BotShieldDashboardAccess 10.1.0.0/16

The dashboard grant covers every page under /dashboard/. The router matches the prefix rather than a list of pages, so a page added in a later release is covered by config that already exists.

Directives accumulate, so keep one network per line with its reason next to it. Two keywords stand in place of the whole list: all serves the endpoint to everyone, and none closes it and refuses a grant inherited from server scope. Neither can be combined with addresses; that is a config-time error rather than a silent resolution, because these lists get edited by commenting lines in and out and a leftover all above a careful CIDR list should not quietly win.

Matching is on the same client address the module scores, so mod_remoteip applies exactly as it does to Apache's own Require ip.

Clearing a flagged address

Directive	Syntax	Default	Scope
BotShieldAdminAccess	<addr|cidr>... | all | none	closed	server / vhost

Who may clear flags from an address. Same grammar and the same closed-until-named rule as the two above, and deliberately a third directive rather than a share of either: the dashboard is a page you read, this changes state. An operator who opens the dashboard to a monitoring host has said nothing about who may unflag.

BotShieldAdminAccess 10.1.0.7

POST <prefix>/admin/unflag
  addr=<ip|cidr>          required
  flags=<name[,name...]>  optional; omitted drops the entry entirely

Requires an X-BotShield-Unflag header, whose value is ignored. The ACL matches an address, an operator's browser sits at an allowed address, and a browser visits pages -- without the header, any page that browser loaded could POST here and be obeyed. No cross-origin form can set a header, so requiring one that no form produces means the request was built deliberately.

$ curl -sS -H 'X-BotShield-Unflag: 1' \
    --data 'addr=203.0.113.0/24' \
    https://example.org/botshield/admin/unflag
cleared 2

Replies cleared <n> in plain text and logs a NOTICE naming the caller, the target, the flags and the count. cleared 0 is a real answer -- the address was not flagged -- and is worth printing rather than treating as an error, because believing an address is flagged is the usual reason for being here.

There is no ns= parameter, though the surface is otherwise per-vhost. The ACL is per-server and namespaces are per-vhost, so honouring a caller-supplied namespace would let a host granted admin on one vhost clear another vhost's reputation. The namespace acted on is the one the request arrived in.

Two things about ranges are worth knowing before typing one:

• An IPv4 /24 clears every flagged address inside it, found by walking the table. The table is hashed on the whole address, so a prefix has no bucket to probe and neighbouring addresses land nowhere near each other. The walk is bounded by table capacity and runs only when asked.
• An IPv6 target is masked to BotShieldIPv6Prefix before matching, because that is the granularity flags are stored at. Asking for a /128 when storage is /64 acts on the /64 -- wider than asked. The reply reports the prefix acted on.

Matching for this directive is on the same client address as the other two, so mod_remoteip applies here as well — the same way it applies to Apache's own Require ip, which compares the forwarded address rather than the address the connection came from.

What makes that safe is RemoteIPTrustedProxy. mod_remoteip honours X-Forwarded-For only from proxies named there, so on a correctly configured server the address this ACL compares is not one the client can choose. Two things are worth checking before granting a write: that RemoteIPHeader is never set without naming the trusted proxies, and that each proxy named overwrites the header rather than appending to a value the client supplied. Get either wrong and the forwarded address becomes attacker-controlled — for this directive, for the two above it, and for Require ip alike.

Comparing the connection address instead is not the safer alternative it sounds like. Behind a reverse proxy that address is the proxy's, and identical for every client, so such a list would either admit everyone who reaches through the proxy or nobody at all. It is only meaningful on a directly exposed server, and it would make this the one address list in the file that means something different from all the others.

Where an IP list is not enough on its own — and for a surface that changes state it may well not be — the answer is a real credential rather than a different address: wrap the path in a <Location> and add Require valid-user or client certificates. That composes with this directive instead of replacing it, since this one decides whether the endpoint is served at all.

Why this is not a <Location>

Require ip on these paths has a failure mode worth knowing. Each authz denial writes an AH01630 line to the error log, the dashboard auto-refreshes every 30 seconds, and a fail2ban jail watching that log counts five denials in two minutes as an attack. On a HubZero host the shipped apache-hz_access_denied jail then bans the viewer at the firewall for ten hours — off the whole site, not just the dashboard. That happened on geodynamics.org on 2026-08-08.

These directives refuse inside the module and write nothing to the error log, so there is nothing for a jail to count. Verified by probe: eight refused dashboard requests produced zero error-log lines from the module.

No <Location> block is needed to make these endpoints reachable. They sit under the vhost's DocumentRoot as far as the directory walk is concerned, so whatever grants the document root grants them, and the module applies the ACL after that. A Require all granted written specifically for them does nothing.

They deliberately stop at an IP list. Apache already has authentication and authorization, they compose by wrapping the path in a <Location>, and a second half-implementation inside this module would be the worse of the two. The IP list is the part Apache cannot supply on the module's behalf, because the module has to decide whether to serve the endpoint at all. To add a password, compound the two — Apache decides whether the request reaches the module, the module decides whether the endpoint is served:

BotShieldDashboardAccess 10.1.0.0/16

<LocationMatch "^/botshield/dashboard">
    AuthType Basic
    AuthName "BotShield"
    AuthUserFile /etc/httpd/botshield.htpasswd
    Require valid-user
</LocationMatch>

Read the resolved lists back with httpd -t -D DUMP_BOTSHIELD_POLICY, which prints one line per endpoint. A wrong allowlist is otherwise discovered by being locked out of the page you would have used to diagnose it.

Refused requests are recorded in the decision log as outcome=observe with reason="observe-denied:<surface>". The response is a 404 and the access-log line is suppressed like every other hit on these endpoints, so that line is the only trace a probe leaves.

Decision log

Directive	Syntax	Default	Scope
BotShieldDecisionLog	/path, logs/path, or "|program"	rotating log beside ErrorLog	server / vhost
BotShieldAccessLog	on, off, suppress=<outcome,...>	on	server / vhost / dir

BotShieldAccessLog controls the Apache access-log line for requests BotShield decided on, not this log. See observability for which outcomes are suppressed by default and why.

A module-owned decision log, written directly from the decision path instead of through mod_log_config:

BotShieldDecisionLog logs/botshield.log
BotShieldDecisionLog "|/usr/bin/rotatelogs /var/log/httpd/bs.%Y%m%d 86400"

Relative paths resolve against ServerRoot, exactly like ErrorLog. A value beginning with | is a piped-log spec handed to Apache's own ap_open_piped_log, so rotatelogs and friends work as they do for any other Apache log.

The default is already rotated

With no directive the module builds this itself:

|<sbindir>/rotatelogs -n 7 -L <dir>/botshield.log.current <dir>/botshield.log 100M

Both halves are deduced rather than hardcoded. <sbindir> comes from apxs -q SBINDIR at build time, the same apxs that compiled the module. <dir> is the directory holding the main server's ErrorLog, so the decision log lands beside it.

Rotation is the default rather than a plain file because the fallback is a file that grows without bound, and this log records every request the access log suppresses. On this deployment that is 100 MB about every 38 hours. An unbounded default is a disk-full incident with a long fuse.

ErrorLog is the anchor for a reason found in testing. A fixed logs/botshield.log resolves through ServerRoot, and two instances sharing a ServerRoot then default to the same file — a test instance's rotator opened production's decision log. Two rotatelogs on one -n slot set truncate each other's slots, which is the exact failure that ate a day of this log. Anchoring to ErrorLog means instances that already log separately get separate decision logs; two instances sharing an ErrorLog path would still collide, but their error logs already do.

If rotatelogs is not present the default degrades to an unrotated file and says so with a NOTICE. If it is present but fails to spawn, the module warns and falls back to the same file rather than refusing to start — a convenience default must never be why httpd will not come up. A pipe you configured failing is still a hard error, because you asked for it.

Rotation and the path your monitoring reads

rotatelogs -n N cycles round-robin through logfile, logfile.1 … logfile.N-1. Which file is current changes over the day, and the base name is the live one only until the first rotation. Anything that reads the base name directly is a bug that hides itself: it returns real, correctly formatted, stale data, which reads as a traffic collapse rather than as a mistake.

Pass -L so a fixed path always hard-links to whichever file is open:

BotShieldDecisionLog "|/usr/sbin/rotatelogs -n 7 \
    -L /var/log/httpd/botshield.log.current \
    /var/log/httpd/botshield.log 100M"

Monitoring then reads botshield.log.current and never has to know about the rotation. The link is re-pointed on each rotation, so a reader that opens by path each time always lands on the live file. Use tail -F (follow by name), not tail -f, which holds the old inode across a rotation.

Timestamped names (bs.%Y%m%d) do not have this problem, because the current name is derivable — but they also grow without bound, which is what -n is for. -L is what makes -n safe to monitor.

Why it exists: a CustomLog cannot survive accesslog=off, because mod_log_config serves every CustomLog from the single log_transaction hook that accesslog=off breaks. An owned log is independent of the access log, which is what lets you rapid-rotate the detection log and archive the access log separately. It also records boring passes at full fidelity with no LogLevel change.

One descriptor per vhost, opened at post_config before the children fork; one write per line, no request-path locking. If the log cannot be opened, startup fails — a decision log you asked for and did not get is a silent blind spot.

Optional. Without it the authoritative record remains the error-log mod_botshield: decision ... line, plus whatever CustomLog you wire up. See observability for the line format and the trade-offs between all three routes.

Log-only / staging mode

Folded into BotShieldEnabled (tri-state on / off / logonly). See the Core section above and the staging guide.

App bridge

Directive	Syntax	Default
BotShieldAppFeedback	on|off	off
BotShieldAppClaims	on|off	off (server / vhost / <Location>)
BotShieldAppIntegrationSecretFile	/path	unset (required for either above)

The feedback header is fixed at X-BotShield-Feedback. It is part of the protocol your application writes against, like the verify endpoint's path — not a setting.

BotShieldAppClaims is per-scope: set it for a whole vhost, or narrow it to the paths whose handler actually reads the header. A <Location> that says off overrides a vhost that turned it on, because unset and explicit-off are distinct states.

<VirtualHost *:443>
    BotShieldAppIntegrationSecretFile /etc/botshield/app-integration-secret
    BotShieldAppClaims On

    <Location /static>
        BotShieldAppClaims Off
    </Location>
</VirtualHost>

The strip is not per-scope. Client-supplied X-Botshield-* request headers are dropped from every request in post_read_request, whether or not the scope emits claims — a request matching no <Location> is exactly the one a forged header would be aimed at. The one exception is X-BotShield-Unflag, which the admin endpoint reads and which asserts nothing about a client.

See captcha for the wire format and security model.

Removed: BotShieldAppFeedbackHeader

Removed 2026-09-11; it now fails config parse.

It bought nothing. The name is namespaced enough not to collide, and it never leaves Apache — the module strips it on the way out — so nothing upstream can rewrite it. Forgery is hard because of the HMAC, not because the name is obscure.

What it did buy was a leak. The strip removes only the configured name, so renaming the header while the application still emitted the old one produced a header the module neither read nor stripped: it went to the client carrying the flag vocabulary and a signature. A fixed name cannot fail that way.

Where to next

• Conceptual model and scoring: site model.
• Deployment topology and capacity: deployment.
• Per-family policy semantics: policy.
• Captcha and app-bridge integration: captcha.
• Decision log and metrics: observability.