diff --git a/docs/.vitepress/config.ts b/docs/.vitepress/config.ts index 00bb319..07c2165 100644 --- a/docs/.vitepress/config.ts +++ b/docs/.vitepress/config.ts @@ -55,6 +55,7 @@ export default defineConfig({ text: 'Advanced', items: [ { text: 'Architecture', link: '/advanced/architecture' }, + { text: 'Performance', link: '/advanced/performance' }, { text: 'Dynamic Throttle & Sliding Window', link: '/advanced/dynamic-throttle' }, { text: 'Request Context', link: '/advanced/request-context' }, { text: 'Portable Config', link: '/advanced/portable-config' }, diff --git a/docs/advanced/performance.md b/docs/advanced/performance.md new file mode 100644 index 0000000..02ed1fb --- /dev/null +++ b/docs/advanced/performance.md @@ -0,0 +1,141 @@ +# Performance + +Phirewall runs on every request, so its cost matters. This page shows what a +`decide()` call costs in isolation, what that means under load, and what is +left of it inside a real application. The short version: a typical rule set +costs single-digit **microseconds** per request, OWASP CRS costs a +**millisecond or two**, and a blocked request is cheaper than a served one. + +## Engine cost per request + +Measured with the engine alone: 20,000 `decide()` calls per scenario after +warmup, `InMemoryCache` store, PHP 8.5 with OPcache and PCRE JIT, Apple +M-series laptop. The rule set is the Quick Start shape: one safelist, the +known-scanners blocklist, one path blocklist, one fail2ban, one allow2ban, +one throttle. + +| Scenario | Outcome | Cost per request | Requests/sec (one core) | +|----------|---------|-----------------:|------------------------:| +| No rules at all (engine floor) | pass | 1.6 us | 640,000 | +| 6 rules, nothing matches (normal traffic) | pass | 8.1 us | 123,000 | +| 6 rules, safelist match (early exit) | safelisted | 1.4 us | 740,000 | +| 6 rules, blocklist match: scanner UA | blocked (403) | 1.9 us | 520,000 | +| 6 rules, blocklist match: path | blocked (403) | 3.2 us | 316,000 | +| 6 rules, fail2ban match below threshold | blocked (403) | 7.8 us | 129,000 | +| 6 rules, key already banned | blocked (403) | 5.2 us | 193,000 | +| 6 rules, throttle exceeded | throttled (429) | 8.7 us | 115,000 | + +Two things stand out: + +- **The expensive case is normal traffic**, because a request that matches + nothing is checked against every rule. Matches exit early and are cheaper. +- **Attackers are the cheap case.** A blocklist match costs 2 to 3 us, and a + banned key is decided by a single store lookup. The requests you block never + reach your application, so under attack the firewall saves far more time + than it costs. + +Absolute numbers depend on your hardware and PHP build. Reproduce them on +your own machine with the bundled benchmarks: + +```bash +XDEBUG_MODE=off php examples/20-rule-benchmarks.php # rule scenarios +XDEBUG_MODE=off php examples/13-benchmarks.php # store backends +``` + +## Store cost + +The table above uses the in-process `InMemoryCache`. The counter rules +(fail2ban, allow2ban, throttle, track) are the ones that talk to the +[store](/features/storage): every request pays one ban lookup per ban-capable +rule, and requests a counter rule matches additionally pay a counter write. +The same scenarios per store, measured inside a Docker setup where Redis and +MariaDB run as separate containers: + +| Scenario (6 rules) | InMemory | APCu | Redis | PDO/MySQL | +|--------------------|---------:|-----:|------:|----------:| +| Nothing matches (2 ban lookups) | 6 us | 6 us | 111 us | 110 us | +| Throttle counting a request | 8 us | 8 us | 213 us | 667 us | +| fail2ban match (403, counter write) | 5 us | 6 us | 160 us | 661 us | +| Key already banned (403) | 4 us | 4 us | 52 us | 55 us | + +How to read it: APCu behaves like in-memory - shared counters on one server +at no measurable cost. The network stores pay roughly 50 us per round trip, +so reads (ban lookups, the every-request cost) sit around 0.1 ms for both. +Writes separate them: a Redis counter update is one round trip (~0.2 ms), +MySQL pays transaction and durability overhead (~0.7 ms). Even the worst +case - every request writing a counter through PDO/MySQL - stays under one +millisecond. Since bans decide on the read path, an attack wave hits the +cheap lookup, not the write. + +## OWASP CRS cost + +The [CRS preset](/features/owasp-crs) evaluates a few hundred ModSecurity +rules per request, most of them regular expressions over your request +parameters, so its cost scales with the number and size of arguments: + +| Request shape (paranoia level 1, no match) | Cost per request | +|--------------------------------------------|-----------------:| +| GET without parameters | ~0.1 ms | +| GET with 3 query parameters | ~1.0 ms | +| POST with 8 form fields | ~2.4 ms | +| SQL injection attempt (matches, early exit) | ~0.4 ms | + +Parsing the rule files costs a few milliseconds once; with the +[compiled data cache](/features/owasp-crs#caching) that price is paid on the +first request after a deployment, not on every request. Combining CRS with a +fail2ban preset also pays off here: once a probing client is banned, the CRS +engine no longer runs for it - the request is rejected by the cheap +banned-key lookup from the table above. + +## What that means under load + +Cost per request times requests per second gives the CPU share the firewall +needs. For the two typical setups: + +| Sustained load | 6 rules (~8 us) | CRS PL1 (~1 ms) | +|----------------|----------------:|----------------:| +| 10 req/s | 0.008% of one core | 1% of one core | +| 100 req/s | 0.08% of one core | 10% of one core | +| 1,000 req/s | 0.8% of one core | one full core | + +Rule-of-thumb: counting rules are free at any realistic load; give CRS a +thought once you serve hundreds of uncached requests per second, and scope it +to the routes that need it if you do. + +## Inside a real application + +Measured end to end on a TYPO3 site (Docker, Apache + mod_php 8.4, cached +page, median of 400 requests per run), once per store: + +| Scenario | APCu | Redis | PDO/MySQL | +|----------|-----:|------:|----------:| +| Firewall middleware removed | 9.1 ms | | | +| Firewall active, 6 rules, nothing matches | 9.6 ms | 9.9 ms | 10.0 ms | +| Blocklist match (scanner user agent, 403) | 6.7 ms | | | +| Banned key (403) | 5.4 ms | 6.4 ms | 5.9 ms | + +The no-match overhead on a cached page is well under a millisecond, and the +store differences drown in run-to-run noise - even with every ban lookup +going through MySQL. On uncached pages the overhead disappears entirely next +to rendering time. And the blocked cases respond *faster* than the page +itself: the request is stopped in the middleware before routing, rendering, +or content queries run. + +## Keeping it fast + +- **OPcache on** (it is on virtually every production setup): the firewall is + plain PHP and profits like the rest of your code. +- **PCRE JIT on** (`pcre.jit=1`, the PHP default): every regex filter and the + CRS `@rx` operators rely on it. See + [PCRE JIT](/features/owasp-crs#pcre-jit). +- **Compiled data cache** for preset packages, so CRS and bad-IP data are not + re-parsed per request. See + [Caching expensive preset data](/advanced/presets#caching-expensive-preset-data). +- **Pick the store deliberately**: APCu for a single server (as fast as + in-memory), Redis when counters and bans must be shared (~0.1 ms of ban + lookups per request, ~0.2 ms per counter write). PDO/MySQL works without + extra infrastructure and stays under a millisecond even on counter writes, + but it puts firewall traffic on your database. +- **Scope expensive rules.** A throttle with a `scope` filter skips its + counter work for requests outside the scope, and CRS can be registered for + the routes that actually take user input. diff --git a/docs/faq.md b/docs/faq.md index 827febc..2ba4823 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -330,7 +330,7 @@ Enable diagnostic headers: $config->enableDiagnosticsHeaders(); ``` -The CRS matcher attaches an `X-Phirewall-Owasp-Rule` header with the matched rule ID to blocked responses. This works wherever the matcher decides the block: as a blocklist rule and as a Fail2Ban filter. Any matcher can ship such headers via the `diagnostic_headers` metadata key on its `MatchResult`. +The CRS matcher attaches an `X-Phirewall-Owasp-Rule` header listing every matched rule ID (capped at 10, then `,+N`) and an `X-Phirewall-Owasp-Score` header (`score/threshold`) to blocked responses. This works wherever the matcher decides the block: as a blocklist rule and as a Fail2Ban filter. Any matcher can ship such headers via the `diagnostic_headers` metadata key on its `MatchResult`. ::: warning Only enable this in development or staging. In production, it reveals information about your security rules to potential attackers. @@ -388,7 +388,8 @@ Enable `$config->enableResponseHeaders()` and check the response headers on bloc |--------|-------| | `X-Phirewall` | Block type: `blocklist`, `throttle`, `fail2ban`, or `allow2ban` | | `X-Phirewall-Matched` | Name of the rule that triggered the block | -| `X-Phirewall-Owasp-Rule` | OWASP rule ID (only if `enableDiagnosticsHeaders()` is active) | +| `X-Phirewall-Owasp-Rule` | Matched OWASP rule IDs, comma-separated (capped at 10, then `,+N`; only if `enableDiagnosticsHeaders()` is active) | +| `X-Phirewall-Owasp-Score` | Accumulated CRS anomaly score as `score/threshold` (only if `enableDiagnosticsHeaders()` is active) | ::: info These headers are disabled by default. Call `$config->enableResponseHeaders()` to enable them for debugging. diff --git a/docs/features/owasp-crs.md b/docs/features/owasp-crs.md index 3eeffd0..3112248 100644 --- a/docs/features/owasp-crs.md +++ b/docs/features/owasp-crs.md @@ -38,16 +38,16 @@ use Flowd\Phirewall\Store\InMemoryCache; $config = new Config(new InMemoryCache()); -// Block requests matching any active CRS rule at paranoia level 1. +// Block requests whose accumulated CRS anomaly score reaches the threshold (default 5). $config = $config->with(Presets::blocklist(ParanoiaLevel::Level1)); ``` -Want to also ban repeat offenders? Use the Fail2Ban preset instead. A CRS -match is malicious by definition, so from 0.8 both presets block every match -with `403`; the difference is that the Fail2Ban preset additionally **bans** -the key after the threshold. A banned attacker is then blocked by a cheap ban -lookup (the CRS engine no longer runs for them), and the ban is observable via -`Fail2BanBanned` and mirrorable to your web server: +Want to also ban repeat offenders? Use the Fail2Ban preset instead. It blocks +the same scoring requests with `403` and additionally **bans** the client key +once it produced `threshold` such requests within `period` seconds. A banned +attacker is then blocked by a cheap ban lookup (the CRS engine no longer runs +for them), and the ban is observable via `Fail2BanBanned` and mirrorable to +your web server: ```php use Flowd\PhirewallPresetOwaspCrs\ParanoiaLevel; @@ -62,6 +62,231 @@ See the [package README](https://github.com/flowd/phirewall-preset-owasp-crs) fo preset API, paranoia-level guidance, and how the bundled rules are imported and kept up to date. +## Anomaly Scoring + +Since preset package 0.5, evaluation follows the CRS anomaly-scoring model instead of +blocking on the first match: every matching rule contributes its `severity` score +(CRITICAL 5, ERROR 4, WARNING 3, NOTICE 2; rules without a recognizable severity are +scored as CRITICAL), and the request is blocked once the accumulated score **reaches** +the threshold (`score >= threshold`, default 5, the CRS standard inbound threshold). + +In practice most bundled rules are CRITICAL and still block on their own; the +WARNING-level rules (for example the `942430` restricted-character checks, a classic +source of false positives on marketing parameters) no longer block alone - two of them +together do. `anomalyThreshold: 1` restores block-on-first-match behaviour: + +```php +$config = $config->with(Presets::blocklist(ParanoiaLevel::Level1, anomalyThreshold: 1)); +``` + +Three decisions bypass the threshold and block immediately (fail closed): a variable +truncated at the collection cap, a single value longer than the per-value inspection +limit (default 2048 bytes, see [Per-Value Length Cap](#per-value-length-cap)), and a +value that triggers a PCRE engine error. +Scores never accumulate across requests; the Fail2Ban preset counts a request toward +the ban whenever CRS blocks it - when its score reaches the threshold or the request +fails closed. + +A request blocked on its anomaly score - or by a rule-level fail-closed outcome (a +capped variable, an oversized value, or a PCRE subject error) - carries +`owasp_anomaly_score`, `owasp_anomaly_threshold`, `owasp_rule_ids` (comma-separated) +and `owasp_rule_id` (first match); it also carries `msg`, `owasp_matched_variable` +(the target the first matching rule fired on, e.g. `REQUEST_HEADERS:User-Agent` - +the member name keeps the casing the client sent), `owasp_matched_value` (the +value the rule fired on - readable so the match can be understood; sanitized, +length-bounded and `[redacted]` for credential targets such as cookie values and +`Authorization`-type headers) and `owasp_log_data` when the first matching rule +provides them, and `owasp_fail_closed` on a fail-closed block +(there `owasp_matched_variable` names the variable that failed closed). A +block from an engine-internal fault under a fail-closed policy (`useFailOpen(false)`) +instead carries only `owasp_anomaly_threshold` and `owasp_fail_closed`. + +**Fail-open policy.** An unexpected engine-internal fault during evaluation - for example a +value manipulator that throws - is governed by the firewall's fail-open setting. Under the +default fail-open policy the fault propagates to the core's error handling and the request is +allowed (a `FirewallError` event is dispatched); when the firewall is configured fail-closed +(fail-open disabled), the matcher blocks the request instead. This is separate from the +rule-level fail-closed decisions above, which always block regardless of the policy. + +## Tuning False Positives + +### Target Exclusions + +Marketing and tracking parameters (`utm_*`, `fbclid`, ...) regularly carry values that +look like attack payloads to CRS rules. Instead of disabling whole rules, exclude the +parameter from inspection - globally, per rule id (CRS `SecRuleUpdateTargetById` +style) or per rule tag. Both presets accept a `configure` closure that receives the +matcher before the first request: + +```php +use Flowd\PhirewallPresetOwaspCrs\Engine\CoreRuleSetMatcher; + +$config = $config->with(Presets::blocklist( + ParanoiaLevel::Level1, + configure: static function (CoreRuleSetMatcher $matcher): void { + $matcher->excludeTarget('ARGS:/^utm_/'); // all rules ignore utm_* values + $matcher->excludeTarget('ARGS_NAMES:/^utm_/'); // ... and the utm_* parameter names + $matcher->excludeTargetById(942431, 'ARGS:fbclid'); // one rule ignores one parameter + $matcher->excludeTargetByTag('attack-sqli', 'ARGS:comment'); + }, +)); +``` + +Selector forms: bare variable (`ARGS`), exact name (`ARGS:utm_source`) or name pattern +(`ARGS:/^utm_/`). Header names match case-insensitively, argument and cookie names +case-sensitively. Exclusions are runtime tuning: they never enter the compiled-data +cache and cannot lift the collection cap. They also do not rewrite the raw +`QUERY_STRING`/`REQUEST_URI` values - the few rules inspecting those still see the +full string; use `disable($ruleId)`, a manipulator, or a bare-variable exclusion +naming the variable that rule actually inspects +(`excludeTargetById(920260, 'REQUEST_URI')`, but `excludeTargetById(931110, +'QUERY_STRING')` since 931110 inspects the query string). A selector naming a +variable the rule does not target is accepted but silently does nothing. + +::: tip Use the CRS 4 tags +A tag that no loaded rule carries makes a tag-scoped exclusion a silent no-op - the +classic case is a CRS 3 tag (`OWASP_CRS/WEB_ATTACK/SQL_INJECTION`) against the +bundled CRS 4 snapshot, whose tags are `attack-sqli` and friends. With a PSR-3 +logger on the matcher, such a tag logs a `warning` once the rules are loaded; +this covers `excludeTargetByTag()` and the tag-scoped CRS exclusion syntax alike. +::: + +### Conditional Exclusions: Validate the Value First + +Every exclude method accepts a `when:` condition - the selected entry is only +excluded while the condition approves its value. That turns a blanket exclusion into +a validated one: a parameter is skipped when it provably carries a legitimate value +and stays fully inspected otherwise. The classic case is a signed token that trips +character-class rules: + +```php +$matcher->excludeTargetByTag( + 'attack-sqli', + 'ARGS:token', + when: static fn (string $variable, ?string $name, string $value): bool + => $jwtValidator->isValid($value), +); +``` + +The condition receives `(string $variable, ?string $name, string $value, +ServerRequestInterface $request)` - the same argument order as a manipulator - +and returns `true` to exclude; a closure declaring fewer parameters ignores the +rest. The request enables context-dependent validation (per-host issuers, +path-scoped rules, comparing against another header). Implement +`TargetExclusionConditionInterface` for a reusable validator. It runs only for +entries the selector matches, and its exceptions propagate like manipulator +exceptions, governed by the failure policy (`useFailOpen()`). + +::: warning Validate strictly +Everything the condition approves is invisible to the rules in scope. Verify the +signature and parse the full format - a shape-only check ("looks like a JWT") +invites attackers to wrap payloads in that shape. +::: + +### CRS Rule-Exclusion Syntax + +`applyRuleExclusions()` and `applyRuleExclusionsFromFile()` - on `CoreRuleSet` and +`CoreRuleSetMatcher` (queued until the rules load, validated eagerly) - accept the +CRS rule-exclusion syntax, so existing ModSecurity tuning files can be reused: + +```php +$config = $config->with(Presets::blocklist( + ParanoiaLevel::Level1, + configure: static function (CoreRuleSetMatcher $matcher): void { + $matcher->applyRuleExclusions(<<<'CONF' + # Configure-time directives: apply once, to the rules already loaded + SecRuleRemoveById 942440 "942430-942432" + SecRuleUpdateTargetById 942100 "!ARGS:search" + SecRuleUpdateTargetByTag attack-sqli "!ARGS:/^utm_/" + + # Runtime exclusion rule: evaluated before the scoring rules on every + # request; its ctl: exclusions apply only to requests it matches. + SecRule REQUEST_URI "@beginsWith /api/webhooks/" \ + "id:10001,phase:1,pass,nolog,\ + ctl:ruleRemoveTargetByTag=attack-sqli;ARGS:payload" + CONF); + }, +)); +``` + +Supported forms: `SecRuleRemoveById` (ids and `from-to` ranges), `SecRuleRemoveByTag` +(exact tag, not a regex), `SecRuleUpdateTargetById` / `SecRuleUpdateTargetByTag` +(negated `!TARGET` removals only), and runtime `SecRule` exclusions with +`ctl:ruleRemoveById`, `ctl:ruleRemoveByTag`, `ctl:ruleRemoveTargetById` or +`ctl:ruleRemoveTargetByTag`. + +Anything the engine cannot evaluate faithfully fails eagerly with an +`InvalidArgumentException` instead of arming a weaker or dead exclusion: chained +rules, unsupported condition operators or variables, target additions and malformed +directives all throw. Unknown directives (`SecMarker`, ...) and other `ctl:` options +(`ctl:ruleEngine`, ...) are skipped. Like every exclusion this is runtime tuning and +never enters the compiled-data cache. + +::: tip Prefer `when:` over `ctl:` shape checks +A runtime exclusion's condition can only pattern-match (`@rx` and friends), so it +can check that a value *looks like* a JWT but not that it *is* one - an attacker can +wrap a payload in the approved shape. When the value can be validated in PHP, prefer +a conditional exclusion (`when:`) that verifies the signature. +::: + +### Manipulators (advanced) + +A manipulator transforms collected values before rules match against them - the escape +hatch when excluding a whole parameter is too broad. Returning an empty string removes +the value from inspection: + +```php +$matcher->addManipulator( + static fn (string $variable, ?string $name, string $value): string + => $name === 'fbclid' ? '' : $value, +); +$matcher->addManipulatorById(942431, $manipulator); // scoped to one rule +``` + +Closure manipulators receive the request as an optional fourth argument +`(string $variable, ?string $name, string $value, ServerRequestInterface $request)` - +the same order as a `when:` exclusion condition; declare it when the +transformation depends on request context. The three-parameter +`RequestValueManipulatorInterface` is unchanged. + +Manipulators run after exclusions; global manipulator results are computed once +per variable per request and shared across all rules, per-rule manipulators +specialize from that shared result. + +::: warning Manipulators weaken detection +Whatever a manipulator removes or rewrites is invisible to every rule it applies to - +including real attack payloads hidden inside the removed content. Prefer target +exclusions; keep manipulators as narrow as possible. Exceptions thrown by a +manipulator propagate. +::: + +### Match Logging + +Pass a PSR-3 logger to either preset (or to `CoreRuleSetMatcher`) to log every rule +match at `info` level - **including sub-threshold matches on requests that pass**. +Those entries are the tuning signal: watch them to find false-positive patterns before +scores ever accumulate to a block, then add a target exclusion. Blocked requests +additionally log a `warning` with total score, threshold and all matched rule ids. + +```php +$config = $config->with(Presets::blocklist(ParanoiaLevel::Level1, logger: $logger)); +``` + +The per-match log context carries `rule_id`, `severity`, `anomaly_score`, +`paranoia_level`, `matched_variable` (e.g. `ARGS:utm_content`), `matched_value` +(the value the rule fired on; `[redacted]` for credential targets), `msg`, `fail_closed`, +`method`, `path` and `log_data` - the rule's CRS `logdata:` template expanded with the +matched data (`%{TX.0}`, `%{MATCHED_VAR_NAME}`, `%{MATCHED_VAR}`); the warning context +carries `total_score`, `anomaly_threshold`, `rule_ids`, `fail_closed`, `method` and +`path`. Attacker-controlled context values are sanitized and length-bounded. + +When the matched target carries a credential - a cookie or an `Authorization`, `Cookie`, +`Proxy-Authorization`, `X-Api-Key` or `X-Auth-Token` header - the value (and its `%{TX.n}` +captures) is replaced with `[redacted]`, so secrets never reach the log or the +`owasp_log_data` metadata. The target name (`%{MATCHED_VAR_NAME}`, e.g. +`REQUEST_COOKIES:session`) is kept, so a redacted match still tells you which parameter to +exclude. + ## Writing Your Own Rules You are not limited to the bundled CRS snapshot - the SecRule engine can parse and @@ -159,6 +384,42 @@ $skipped = $report['skipped']; // int - Lines that were skipped Every factory accepts an optional `$maxValuesPerCrsVariable`: a positive-int cap on how many values are collected per CRS variable per request. It bounds the evaluation cost of count-unbounded, attacker-controlled variables such as `ARGS` (a CPU-DoS guard). The default (`null`) derives the cap from twice PHP's `max_input_vars`, falling back to 2000 when the directive is unset or non-positive, so a request PHP can fully parse is never falsely truncated. When a variable *is* truncated at the cap, rules targeting it fail closed and treat the request as a match, so padding a payload past the cap cannot evade a rule. A value `< 1` throws `InvalidArgumentException`. +### Per-Value Length Cap + +The count cap above bounds *how many* values are collected. A second cap bounds +*how long a single value* may be: a collected value longer than +`CoreRule::MAX_INSPECTABLE_VALUE_LENGTH` (default **2048 bytes**) is treated as +un-inspectable, and the rule **fails closed** (blocks) - the same contract as the +count cap. One mechanism covers three concerns at once: + +- **Evasion.** A payload placed past the limit (`?q=` + 9 KB of filler + `UNION SELECT …`) can no longer slip through - the oversized value blocks instead of being partially inspected. +- **ReDoS.** No value longer than the cap ever reaches the regex engine, so worst-case `@rx` backtracking is bounded by the cap, not by the request size. +- **Unbounded scan.** The phrase/substring operators (`@contains`, `@pm`, `@pmFromFile`, …) never scan an arbitrarily large value. + +Tune it per rule set with `CoreRuleSet::setMaxInspectableValueLength(int $bytes)` +or `CoreRuleSetMatcher::setMaxInspectableValueLength(int $bytes)` (the latter is +reachable through the presets' `configure:` closure and queued until the rules +load). Both validate `$bytes >= 1` and return `$this` for chaining: + +```php +use Flowd\PhirewallPresetOwaspCrs\Engine\CoreRuleSetMatcher; + +$config = $config->with(Presets::blocklist( + ParanoiaLevel::Level1, + configure: static fn (CoreRuleSetMatcher $matcher) => $matcher->setMaxInspectableValueLength(8192), +)); +``` + +Choosing the value is a trade-off: + +- **Lower** (e.g. 1024) - **pro:** tighter worst-case regex time and a smaller per-value CPU budget. **con:** more false-positive blocks, because a value over the cap is blocked even when it is harmless. +- **Higher** (e.g. 8192) - **pro:** fewer false positives on legitimately large single values, notably long tokens in a `Cookie` or `Authorization` header and base64 fields. **con:** a larger subject reaches the regex engine, so the worst-case backtracking cost the cap bounds grows with it (roughly cubic for the pathological CRS XSS patterns). + +The default of 2048 balances the two: it bounds a pathological `@rx` subject to +roughly a second of worst-case work while passing typical request values. Raise +it if your application legitimately sends large single fields or tokens; lower it +for a stricter CPU bound when your traffic has no large single fields. + ## Supported SecRule Syntax Phirewall supports a subset of the ModSecurity SecRule language: @@ -178,6 +439,12 @@ Phirewall supports a subset of the ModSecurity SecRule language: | `REQUEST_COOKIES` | All cookie values | | `REQUEST_COOKIES_NAMES` | Names of all cookies | +Collection variables also accept **named selectors**: `REQUEST_HEADERS:User-Agent` +inspects only that header, and negated selectors such as `!REQUEST_HEADERS:Cookie` or +`!ARGS_NAMES:/^utm_/` exclude members from the rule's bare selector of the same +variable. Selectors of unsupported variables (`XML:/*`, `REQUEST_BODY`) collect +nothing; the rule evaluates against its supported targets only. + ### Operators | Operator | Syntax | Description | @@ -185,21 +452,39 @@ Phirewall supports a subset of the ModSecurity SecRule language: | `@rx` | `@rx pattern` | PCRE regular expression match | | `@contains` | `@contains text` | Case-insensitive substring match | | `@streq` | `@streq text` | Case-insensitive exact string match | -| `@startswith` | `@startswith text` | Case-insensitive prefix match | -| `@beginswith` | `@beginswith text` | Alias for `@startswith` | -| `@endswith` | `@endswith text` | Case-insensitive suffix match | +| `@beginsWith` | `@beginsWith text` | Case-insensitive prefix match | +| `@startsWith` | `@startsWith text` | Alias for `@beginsWith` (phirewall extension; not a ModSecurity operator) | +| `@endsWith` | `@endsWith text` | Case-insensitive suffix match | | `@pm` | `@pm word1 word2` | Phrase match (case-insensitive substring match against any of the listed phrases) | | `@pmFromFile` | `@pmFromFile file.txt` | Phrase match from a file (one phrase per line) | +Operator names themselves are matched case-insensitively (`@beginswith` parses +just as well); this page uses the canonical ModSecurity casing. + +The string operators (`@streq`, `@contains`, `@beginsWith`, `@endsWith`) match +case-insensitively. In ModSecurity these operators are case-sensitive and case +folding comes from a `t:lowercase` transformation on the target; because this engine +ignores transformations, folding case in the operators reproduces that common CRS +pattern. A rule that genuinely wanted case-sensitive matching without `t:lowercase` +is matched case-insensitively instead (a safe over-match, never an under-match). + ### Actions | Action | Description | |--------|-------------| | `id:N` | Rule ID (required, must be unique) | | `phase:N` | Processing phase (currently informational) | -| `deny` | Block the request (required for the rule to trigger blocking) | -| `block` | Alias for `deny` - both trigger blocking | -| `msg:'text'` | Human-readable description for logging | +| `deny` | Make the rule score/block (required for the rule to participate) | +| `block` | Alias for `deny` | +| `msg:'text'` | Human-readable description for logging and metadata | +| `severity:'LEVEL'` | Anomaly score contribution: CRITICAL 5, ERROR 4, WARNING 3, NOTICE 2 (missing/unknown scores as CRITICAL) | +| `logdata:'template'` | Log template expanded on a match (`%{TX.0}`, `%{MATCHED_VAR_NAME}`, `%{MATCHED_VAR}`) | +| `tag:'name'` | Rule tags (repeatable); `paranoia-level/N` sets the level, tags drive `excludeTargetByTag()` | +| `ctl:ruleRemove*` | Honored only in exclusion text applied via [`applyRuleExclusions()`](#crs-rule-exclusion-syntax); ignored in normal rule loading | + +Unlisted actions (`t:`, `setvar:`, `chain`, ...) are ignored during normal rule +loading; a rule whose evaluation would depend on them (e.g. a chained rule) is +dropped at import of the bundled snapshot. ### Line Continuation @@ -223,17 +508,19 @@ SecRule ARGS "@rx (?i)\bunion\b.*\bselect\b" "id:942100,phase:2,deny,msg:'SQLi'" ### Tuning the Bundled Snapshot -The presets from the [Quick Start](#quick-start) are fixed rule bundles. To tune the bundled -CRS snapshot (for example, to drop a false-positive-prone rule), load it as a mutable -`CoreRuleSet` via `Presets::coreRuleSet()` and wire it yourself: +The presets from the [Quick Start](#quick-start) accept a `configure` closure for +[exclusions and manipulators](#tuning-false-positives). For full manual control, load +the bundled snapshot as a mutable `CoreRuleSet` via `Presets::coreRuleSet()` and wire +it yourself: ```php use Flowd\PhirewallPresetOwaspCrs\Engine\CoreRuleSetMatcher; use Flowd\PhirewallPresetOwaspCrs\ParanoiaLevel; use Flowd\PhirewallPresetOwaspCrs\Presets; -$rules = Presets::coreRuleSet(ParanoiaLevel::Level2); -$rules->disable(942100); // SQLi via libinjection, if it false-positives for your app +$rules = Presets::coreRuleSet(ParanoiaLevel::Level2) + ->excludeTarget('ARGS:/^utm_/'); +$rules->disable(942430); // restricted SQL character anomaly, if it false-positives for your app $config->blocklists->addRule(new BlocklistRule('owasp', new CoreRuleSetMatcher($rules))); ``` @@ -289,14 +576,18 @@ $config->enableResponseHeaders(); $config->enableDiagnosticsHeaders(); ``` -When an OWASP rule blocks a request, the response includes: +When the accumulated anomaly score blocks a request, the response includes: ``` X-Phirewall: blocklist X-Phirewall-Matched: owasp -X-Phirewall-Owasp-Rule: 942100 +X-Phirewall-Owasp-Rule: 942430,942431 +X-Phirewall-Owasp-Score: 6/5 ``` +`X-Phirewall-Owasp-Rule` lists every matched rule id (capped at 10, then `,+N`); +`X-Phirewall-Owasp-Score` is `score/threshold`. + ::: info `X-Phirewall` and `X-Phirewall-Matched` require `enableResponseHeaders()`. The `X-Phirewall-Owasp-Rule` header is controlled independently by `enableDiagnosticsHeaders()` (`enableOwaspDiagnosticsHeader()` is a deprecated alias): the CRS matcher declares it via the generic `diagnostic_headers` metadata key on its `MatchResult`, so it also appears when the matcher is used as a Fail2Ban filter. ::: @@ -353,52 +644,38 @@ SecRule REQUEST_URI "@rx (?i)(%2e%2e%2f|%2e%2e/)" \ ## Production Configuration -A production rule set covering the main attack categories: +For production, use the bundled OWASP CRS preset - a filtered, tested snapshot of the full +Core Rule Set covering every attack category - rather than hand-written rules. Pair it with a +compiled-data cache so the rules are parsed once per deployment instead of on every request, +and a PSR-3 logger so you can watch sub-threshold matches before they ever block: ```php use Flowd\Phirewall\Config; -use Flowd\Phirewall\Config\Rule\BlocklistRule; -use Flowd\PhirewallPresetOwaspCrs\Engine\CoreRuleSetMatcher; -use Flowd\PhirewallPresetOwaspCrs\Engine\SecRuleLoader; use Flowd\Phirewall\Store\RedisCache; +use Flowd\Phirewall\Support\CompiledDataCache; +use Flowd\PhirewallPresetOwaspCrs\Presets; +use Flowd\PhirewallPresetOwaspCrs\ParanoiaLevel; use Predis\Client as PredisClient; $redis = new PredisClient(getenv('REDIS_URL') ?: 'redis://localhost:6379'); $config = new Config(new RedisCache($redis)); +$config->setCompiledDataCache(new CompiledDataCache('/var/cache/phirewall')); -$rules = SecRuleLoader::fromString(<<<'CRS' -# ── SQL Injection ────────────────────────────────────────── -SecRule ARGS "@rx (?i)(\bunion\b.*\bselect\b|\bselect\b.*\bfrom\b)" \ - "id:942100,phase:2,deny,msg:'SQL Injection'" -SecRule ARGS "@rx ('\s*(or|and)\s*'|'\s*=\s*')" \ - "id:942120,phase:2,deny,msg:'SQL Quote Injection'" - -# ── XSS ─────────────────────────────────────────────────── -SecRule ARGS "@rx (?i)]*>" \ - "id:941100,phase:2,deny,msg:'XSS Script Tag'" -SecRule ARGS "@rx (?i)\bon\w+\s*=" \ - "id:941110,phase:2,deny,msg:'XSS Event Handler'" -SecRule ARGS "@rx (?i)javascript\s*:" \ - "id:941120,phase:2,deny,msg:'XSS JavaScript Protocol'" - -# ── Remote Code Execution ───────────────────────────────── -SecRule ARGS "@rx (?i)(eval|exec|system|shell_exec|passthru)\s*\(" \ - "id:933100,phase:2,deny,msg:'PHP Code Injection'" -SecRule ARGS "@rx (?i)(base64_decode|gzinflate|str_rot13)\s*\(" \ - "id:933110,phase:2,deny,msg:'PHP Obfuscation'" - -# ── Path Traversal ──────────────────────────────────────── -SecRule REQUEST_URI "@rx \.\.\/" \ - "id:930100,phase:2,deny,msg:'Path Traversal'" -SecRule REQUEST_URI "@rx (?i)(%2e%2e%2f|%2e%2e/)" \ - "id:930110,phase:2,deny,msg:'Encoded Path Traversal'" -CRS); +$config = $config->with(Presets::blocklist( + ParanoiaLevel::Level1, // start low; raise only after tuning + anomalyThreshold: 5, // the CRS standard inbound threshold + logger: $logger, // sub-threshold matches are the tuning signal +)); +``` -// Disable rules that cause false positives in your application -// $rules->disable(941110); // XSS Event Handler +Start at paranoia level 1 with the default threshold, watch the log for false-positive +patterns, add [target exclusions](#target-exclusions), and only then raise the paranoia +level - see [Tuning False Positives](#tuning-false-positives). -$config->blocklists->addRule(new BlocklistRule('owasp', new CoreRuleSetMatcher($rules))); -``` +Hand-writing a SecRule set (see [Writing Your Own Rules](#writing-your-own-rules)) is only +for narrow, app-specific checks the CRS does not cover. Do not reimplement SQLi/XSS/RCE +detection by hand: a naive pattern such as `@rx (?i)]*>` misses the large majority +of real payloads that the maintained CRS catches. ## File-Based Rule Management @@ -417,6 +694,20 @@ $report = SecRuleLoader::fromStringWithReport( echo "Parsed: {$report['parsed']}, Skipped: {$report['skipped']}\n"; ``` +Tuning belongs in its own file: keep your +[CRS rule exclusions](#crs-rule-exclusion-syntax) next to the application and +apply them with `applyRuleExclusionsFromFile()` - the file is read eagerly, so +a missing, unreadable or malformed tuning file fails at configuration time, not +on the first request: + +```php +$config = $config->with(Presets::blocklist( + ParanoiaLevel::Level1, + configure: static fn (CoreRuleSetMatcher $matcher) => + $matcher->applyRuleExclusionsFromFile('/etc/phirewall/crs-exclusions.conf'), +)); +``` + ### @pmFromFile Support The `@pmFromFile` operator loads phrase lists from external files. The file path is resolved relative to the rule file's directory: @@ -434,7 +725,7 @@ insert into ``` ::: warning -`@pmFromFile` includes path traversal protection. Paths containing `..` are rejected to prevent loading files outside the rules directory. +`@pmFromFile` confines the operand to the rules directory. Directory traversal (`..`), absolute paths, and stream-wrapper schemes (`file://`, `php://`, ...) are all rejected, so a rule cannot load a file from outside its directory. An operand that cannot be safely resolved fails closed - the rule blocks - rather than being silently skipped, so a misconfigured `@pmFromFile` cannot quietly disable protection. ::: ## Architecture @@ -452,7 +743,7 @@ SecRule ARGS "@rx (?i)union.*select" "id:942100,phase:2,deny,msg:'SQLi'" Variable --------> VariableCollectorFactory --> ArgsCollector ``` -When a rule is constructed, the factories resolve the variable names and operator into concrete strategy instances. On each request, `CoreRule::matches()` collects values via the variable collectors and passes them to the operator evaluator. +When a rule is constructed, the factories resolve the variable names and operator into concrete strategy instances. On each request, `CoreRuleSet::evaluate()` collects values via the variable collectors (applying exclusions and manipulators), passes them to the operator evaluators and accumulates the matching rules' severity scores into a `RuleSetEvaluation`. ### Variable Collectors @@ -477,20 +768,20 @@ Each CRS operator maps to an `OperatorEvaluatorInterface` implementation: | Operator | Evaluator Class | Behavior | |----------|----------------|----------| -| `@rx` | `RegexEvaluator` | PCRE match with auto-delimiters and Unicode mode; values longer than 8 KiB are truncated to 8,192 bytes and the head is still matched (a PCRE engine error fails closed to a match) | +| `@rx` | `RegexEvaluator` | PCRE match with auto-delimiters and Unicode mode; a subject-induced PCRE engine error fails closed. Oversized values are bounded upstream by the [per-value length cap](#per-value-length-cap), so the evaluator no longer truncates. | | `@contains` | `ContainsEvaluator` | Case-insensitive substring search | | `@streq` | `StringEqualEvaluator` | Case-insensitive exact match | -| `@startswith` / `@beginswith` | `StartsWithEvaluator` | Case-insensitive prefix match | -| `@endswith` | `EndsWithEvaluator` | Case-insensitive suffix match | +| `@beginsWith` / `@startsWith` | `StartsWithEvaluator` | Case-insensitive prefix match | +| `@endsWith` | `EndsWithEvaluator` | Case-insensitive suffix match | | `@pm` | `PhraseMatchEvaluator` | Multi-phrase case-insensitive match | | `@pmFromFile` | `PhraseMatchFromFileEvaluator` | Phrase match from file with path traversal protection | Unsupported operators resolve to `UnsupportedOperatorEvaluator`, which never matches (safe no-op). -::: warning ReDoS protection: 8 KiB length guard on `@rx` -`RegexEvaluator` does **not** skip overlength values. A value longer than 8,192 bytes is truncated to that length (dropping a partial trailing UTF-8 sequence) and the **head is still matched** against the pattern. This bounds the PCRE work on unbounded attacker-controlled input - which risks catastrophic backtracking that can freeze the PHP process (ReDoS) - while preventing evasion by padding a payload past the limit. A value that triggers a PCRE engine error (catastrophic backtracking, invalid UTF-8 under `/u`, backtrack/recursion limit) is treated as a **match** (fail-closed), so a malformed payload can never silently disable a rule. +::: warning ReDoS protection: per-value length cap + fail-closed +Catastrophic `@rx` backtracking (which can freeze the PHP process) is bounded by the [per-value length cap](#per-value-length-cap): a value longer than `CoreRule::MAX_INSPECTABLE_VALUE_LENGTH` (default 2048 bytes) never reaches the regex engine - the rule fails closed instead of matching a head window. A subject that still triggers a PCRE engine error *within* the limit (invalid UTF-8 under `/u`, backtrack/recursion limit) also fails closed, so a malformed payload can never silently disable a rule. -In practice, legitimate request values (query parameters, header values, cookie values) are rarely larger than a few kilobytes, so the truncation only affects oversized, likely-hostile input. +Earlier versions truncated an oversized value to an 8 KiB head and matched only that head. That left the 8 KiB head as the peak backtracking-cost point *and* let a payload evade by sitting past the head. The current fail-closed cap replaces that behavior, and its length is configurable - see [Per-Value Length Cap](#per-value-length-cap) for the trade-off between a tighter CPU bound and false-positive blocks on large legitimate values. ::: ### Adding Custom Operators @@ -535,15 +826,18 @@ use Psr\Http\Message\ServerRequestInterface; final readonly class RequestBodyCollector implements VariableCollectorInterface { - /** @return list */ + /** @return list */ public function collect(ServerRequestInterface $serverRequest): array { $body = (string) $serverRequest->getBody(); - return $body !== '' ? [$body] : []; + return $body !== '' ? [['name' => null, 'value' => $body]] : []; } } ``` +Each entry carries the member name it belongs to (parameter, cookie or header name) +so selectors and exclusions can address it; unnamed variables use `name: null`. + ## Performance ### Caching @@ -561,7 +855,7 @@ use Flowd\PhirewallPresetOwaspCrs\ParanoiaLevel; $config->setCompiledDataCache(new CompiledDataCache('/path/to/var/cache/phirewall')); $matcher = CoreRuleSetMatcher::fromRuleFiles(ParanoiaLevel::Level1); -$matcher->disable(942100); // toggles before the first request are queued +$matcher->disable(941110); // toggles before the first request are queued $config->blocklists->addRule(new BlocklistRule('owasp', $matcher)); ``` @@ -573,7 +867,7 @@ $config->blocklists->addRule(new BlocklistRule('owasp', $matcher)); |----------|:------------:|-------| | `@streq` | Low | Simple string comparison | | `@contains` | Low | Substring search | -| `@startswith` / `@endswith` | Low | Prefix/suffix check | +| `@startsWith` / `@endsWith` | Low | Prefix/suffix check | | `@pm` | Medium | Case-insensitive phrase matching (pre-compiled) | | `@rx` | High | PCRE regex (compiled on first use, cached) |