Skip to content

Add permittable:audit — coverage, including what is not covered - #18

Open
VSN2015 wants to merge 1 commit into
masterfrom
feature/audit-task
Open

Add permittable:audit — coverage, including what is not covered#18
VSN2015 wants to merge 1 commit into
masterfrom
feature/audit-task

Conversation

@VSN2015

@VSN2015 VSN2015 commented Sep 4, 2026

Copy link
Copy Markdown
Owner

The blind spot

A controller declaring permit_params :create looks adopted. If it also answers PATCH, that action validates nothing — and nothing in the gem said so:

  • permittable:generate only notices controllers with no contract at all.
  • The OpenAPI exporter documents what exists, not what's missing.

Half-coverage was invisible, which is the worst state for a security-adjacent tool to leave you in: the dashboard says "adopted" and the PATCH endpoint takes anything.

The audit

Permittable::Audit is the fourth reader of the frozen registry, and the first to cross it with the route set:

bin/rails permittable:audit             # the table plus a summary
bin/rails "permittable:audit[strict]"   # ...and exit 1 on any unguarded write action
legacy/invoices
  POST   /legacy/invoices                   create       no contract — ACCEPTS A BODY
orders
  POST   /orders                            create       enforce
  PUT    /orders/{id}                       update       no contract — ACCEPTS A BODY
users
  GET    /users                             index        no contract
  POST   /users                             create       enforce  model: User  unknown: error
  DELETE /users/{id}                        destroy      monitor
  PATCH  /users/{id}                        update       enforce  model: User  unknown: error

7 routed actions: 3 enforced, 1 in monitor mode, 3 without a contract
  2 of those accept a request body — untrusted input reaches the action unchecked
  2 covered actions declare no model:, so no schema-drift guard runs for them

Contracts declared for actions no route reaches (renamed or deleted?):
  users#archive

Three things nothing else tells you

1. Which write actions are unguarded. A GET without a contract is usually fine; a POST without one is untrusted input reaching the action unchecked. Separating those two is the whole point of the — ACCEPTS A BODY marker, and that count is what [strict] exits 1 on. That makes the task a CI gate: no new unguarded write endpoint.

2. Which contracts aren't enforcing yet. Because an audit runs inside the app rather than being exported from it, it can resolve the effective mode — a rule's own mode: first, then your app-wide Permittable.mode — which the OpenAPI exporter deliberately cannot (runtime config isn't contract data). This is the monitor-mode rollout dashboard.

3. Which contracts have gone stale. A contract declared for an action no route reaches is a renamed or deleted action that left its contract behind.

Controllers that never included Permittable are audited too — those are the ones worth finding, so the task collects every ActionController::Base/API descendant, not just the ones answering permit_rule_for.

Design

Plain Ruby over the registry plus { controller:, action:, verb:, path: } route descriptors — the shape OpenAPI.rails_routes already produces — so Permittable::Audit.entries(controllers:, routes:) is unit-testable without Rails, same as the exporter. .summary returns the counts for a CI log, .stale the leftovers, .format the human report.

Verification

  • 209 examples, 0 failures (10 new in spec/audit_spec.rb, written before the implementation), covering catch-all expansion, effective-mode resolution under a changed Permittable.mode, and concern-free controllers
  • RuboCop clean, 29 files
  • docs/comparison.md gains one honest row: no other option in that table reports what isn't covered

@VSN2015 VSN2015 left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of PR #18: permittable:audit provides full visibility into which route endpoints validate parameters. The distinction between body-accepting actions and read endpoints makes the audit actionable.


next unless task_args[:strict]

gap = Permittable::Audit.summary(entries)[:uncovered_with_body]

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the [strict] argument to abort when uncovered_with_body > 0 enables teams to run permittable:audit[strict] as a continuous integration quality gate.

Comment thread lib/permittable/audit.rb
# `Permittable.mode`, so it can resolve what the exporter deliberately
# cannot.
module Audit
module_function

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Limiting BODY_VERBS to POST, PUT, and PATCH appropriately scopes the finding to routes that expect inbound client payloads, avoiding false alarms on standard GET endpoints.

A controller declaring permit_params :create looks adopted. If it also
answers PATCH, that action validates nothing, and nothing in the gem
said so: permittable:generate only notices controllers with no contract
at all, and the OpenAPI exporter documents what exists rather than what
is missing. Half-coverage was invisible.

Permittable::Audit is the fourth reader of the frozen registry, and it
crosses it with the ROUTE SET, so a half-covered controller is as
visible as an uncovered one. Per routed action it reports the rule a
request would actually resolve through, that rule's effective mode,
whether a model: guards its columns, and — for an uncovered action —
whether it accepts a request body, which is the difference between a
gap in a table and untrusted input reaching an action unchecked.

Two things follow from the audit running inside the app rather than
being exported from it. It can resolve the EFFECTIVE mode (a rule's own
mode: first, then the app-wide Permittable.mode) where the OpenAPI
exporter deliberately cannot, which makes it the monitor-mode rollout
dashboard. And `bin/rails "permittable:audit[strict]"` can exit 1 on
the unguarded-write count, which makes it a CI gate: no new unguarded
write endpoint.

It also lists contracts declared for actions no route reaches — a
renamed or deleted action that left its contract behind — and audits
controllers that never included Permittable at all, since those are
the ones worth finding.

Plain Ruby over the registry plus { controller:, action:, verb:, path: }
route descriptors, the shape OpenAPI.rails_routes already produces, so
Audit.entries is unit-testable without Rails.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant