diff --git a/docs/best-practices.md b/docs/best-practices.md index deae63c..224a881 100644 --- a/docs/best-practices.md +++ b/docs/best-practices.md @@ -48,13 +48,12 @@ request, then on the form. The plan behind it is slice |---|---|---|---|---| | MUST | 39 | 0 | 4 | 43 | | SHOULD | 10 | 0 | 0 | 10 | -| SUGGESTED | 9 | 4 | 1 | 14 | -| **All** | **58** | **4** | **5** | **67** | +| SUGGESTED | 10 | 3 | 1 | 14 | +| **All** | **59** | **3** | **5** | **67** | Passing needs every MUST met or N/A, every SHOULD met or explained, and every SUGGESTED -answered. Every MUST and SHOULD is met or N/A today. The four unmet SUGGESTED criteria -(`version_tags`, `warnings_strict`, `dynamic_analysis`, `dynamic_analysis_enable_assertions`) -don't block passing. +answered. Every MUST and SHOULD is met or N/A today. The three unmet SUGGESTED criteria +(`version_tags`, `dynamic_analysis`, `dynamic_analysis_enable_assertions`) don't block passing. What has to happen for the badge, and to keep it, and who does it: @@ -73,7 +72,7 @@ What has to happen for the badge, and to keep it, and who does it: | `description_good` | MUST | Met | The README's first paragraph says what Kotiko does: learn a word in any language and Kotiko slips it into the web pages you already read, in whatever language you read them. https://github.com/ScriptKittyOS/kotiko#readme | `README.md:16-19`; repository description (`gh api repos/ScriptKittyOS/kotiko`) | – | | `interact` | MUST | Met | Obtain: the README's Install section (https://github.com/ScriptKittyOS/kotiko#install); until the store listings and first release exist, the extension is loaded from the repository's extension folder (steps: https://github.com/ScriptKittyOS/kotiko/blob/main/site/src/content/docs/install.mdx). Feedback: issue forms for bugs, ideas, broken sites and wrong words (https://github.com/ScriptKittyOS/kotiko/issues/new/choose), questions in GitHub Discussions (https://github.com/ScriptKittyOS/kotiko/discussions), security reports per SECURITY.md. Contribute: https://github.com/ScriptKittyOS/kotiko/blob/main/CONTRIBUTING.md | `README.md:54-64`, `README.md:93-98`; `site/src/content/docs/install.mdx:48-107`; `.github/ISSUE_TEMPLATE/bug.yml`, `feature.yml`, `site-broken.yml`, `wrong-word.yml`, `config.yml:1-8`; `CONTRIBUTING.md:13` | – | | `contribution` | MUST [URL] | Met | Contributions are GitHub pull requests: pick a slice or issue, comment on it, open a pull request; maintainers review every pull request and CI must pass before it merges. https://github.com/ScriptKittyOS/kotiko/blob/main/CONTRIBUTING.md | `CONTRIBUTING.md:13`, `CONTRIBUTING.md:65-70`; ruleset "main: pull requests with passing CI" (`gh api repos/ScriptKittyOS/kotiko/rulesets/24531669`) | – | -| `contribution_requirements` | SHOULD [URL] | Met | CONTRIBUTING.md lists the requirements: rules that protect users, tests with every change and a regression test with every fix, Conventional Commits, SPDX headers, and the coding standards in docs/CODING_STANDARDS.md. https://github.com/ScriptKittyOS/kotiko/blob/main/CONTRIBUTING.md | `CONTRIBUTING.md:38-97`; `docs/CODING_STANDARDS.md` | – | +| `contribution_requirements` | SHOULD [URL] | Met | CONTRIBUTING.md states what an acceptable contribution needs: the rules that protect users (text-only DOM, no runtime npm code in the extension, no user text or secrets in logs, no secrets in the repository, no network in tests), automated tests with every behaviour change and a regression test with every fix, Conventional Commits, SPDX headers, and the coding standards in docs/CODING_STANDARDS.md, which CI checks. https://github.com/ScriptKittyOS/kotiko/blob/main/CONTRIBUTING.md and https://github.com/ScriptKittyOS/kotiko/blob/main/docs/CODING_STANDARDS.md | `CONTRIBUTING.md:38-97`; `docs/CODING_STANDARDS.md` | – | | `floss_license` | MUST | Met | The code is licensed under Apache-2.0; two bundled data files also carry the MIT and Unicode-3.0 licenses of their sources. The name, logo, icons and illustrations are trademarks under separate terms (LICENSES/LicenseRef-KotikoBrand.txt), not part of the code license. | `LICENSE`; `README.md:100-104`; `REUSE.toml:17-33` | – | | `floss_license_osi` | SUGGESTED | Met | Apache-2.0 (and MIT and Unicode-3.0 for two bundled data files) are OSI-approved. | `LICENSE`; `LICENSES/` | – | | `license_location` | MUST [URL] | Met | https://github.com/ScriptKittyOS/kotiko/blob/main/LICENSE (plus LICENSES/ and REUSE.toml for per-file licensing). | `LICENSE`; `LICENSES/`; `REUSE.toml`; CI runs `reuse lint` (`.github/workflows/ci.yml:143`) | – | @@ -91,7 +90,7 @@ What has to happen for the badge, and to keep it, and who does it: | `repo_public` | MUST | Met | Public git repository on GitHub: https://github.com/ScriptKittyOS/kotiko | `gh api repos/ScriptKittyOS/kotiko`: `"visibility":"public"` | – | | `repo_track` | MUST | Met | git records every change with its author and date. https://github.com/ScriptKittyOS/kotiko/commits/main | `git log` | – | | `repo_interim` | MUST | Met | Every change to main goes through a pull request (a ruleset requires pull requests and eight passing CI checks), and the full history with its interim commits is public, not only releases. https://github.com/ScriptKittyOS/kotiko/pulls?q=is%3Apr | `git log --merges` (44 merges); ruleset 24531669 | – | -| `repo_distributed` | SUGGESTED | Met | git. | – | – | +| `repo_distributed` | SUGGESTED | Met | git, hosted on GitHub: https://github.com/ScriptKittyOS/kotiko | – | – | | `version_unique` | MUST | Met | Each release has its own SemVer version (0.1.0, 0.2.0, next 1.0.0), kept identical in the server, the extension manifest and the release manifest; CI fails if they differ. | `server/mix.exs:11`; `extension/manifest.json:5`; `.release-please-manifest.json`; `node scripts/check-versions.mjs`: all 0.2.0, exit 0 (CI job "versions and licenses", `.github/workflows/ci.yml:131`) | – | | `version_semver` | SUGGESTED | Met | Semantic Versioning; release-please derives versions from Conventional Commits. | `CHANGELOG.md:209,217`; `release-please-config.json`; `.release-please-manifest.json` | – | | `version_tags` | SUGGESTED | Unmet | No release has been tagged in this repository yet. From the first public release (v1.0.0) every release gets a signed vX.Y.Z tag, which the release pipeline verifies before it builds anything, and a ruleset lets only admins create, move or delete release tags. | `git ls-remote --tags origin` is empty; `gh release list` is empty; ruleset "release tags: only admins create, move or delete"; `.github/workflows/release.yml` | Maintainer adds the signing key to `.github/allowed_signers`, merges release PR #7 and tags `v1.0.0` | @@ -103,7 +102,7 @@ What has to happen for the badge, and to keep it, and who does it: | Criterion | Level | Status | Answer | Evidence | Needs | |---|---|---|---|---|---| | `report_process` | MUST [URL] | Met | Bugs are reported as GitHub issues, with forms for bugs, broken sites and wrong words: https://github.com/ScriptKittyOS/kotiko/issues/new/choose | `.github/ISSUE_TEMPLATE/bug.yml`, `site-broken.yml`, `wrong-word.yml`; `has_issues` true | – | -| `report_tracker` | SHOULD | Met | GitHub Issues. https://github.com/ScriptKittyOS/kotiko/issues | `.github/ISSUE_TEMPLATE/` | – | +| `report_tracker` | SHOULD | Met | GitHub Issues, with forms for bugs, ideas, broken sites and wrong words: https://github.com/ScriptKittyOS/kotiko/issues | `.github/ISSUE_TEMPLATE/` | – | | `report_responses` | MUST | Met | No bug reports have been submitted yet (the repository opened on 2026-10-06). Maintainers take a weekly triage turn: every new issue gets a label and a first response within 7 days (GOVERNANCE.md). | `GOVERNANCE.md:41`; GraphQL `issues.totalCount` is 0 (2026-10-06) | Keep answering within 7 days | | `enhancement_responses` | SHOULD | Met | No enhancement requests have been submitted yet; the same 7-day triage applies to them (GOVERNANCE.md). | As above | As above | | `report_archive` | MUST [URL] | Met | GitHub Issues keeps every report and reply, searchable: https://github.com/ScriptKittyOS/kotiko/issues?q=is%3Aissue | – | – | @@ -119,15 +118,15 @@ What has to happen for the badge, and to keep it, and who does it: | `build_common_tools` | SUGGESTED | Met | Mix (Elixir's standard build tool) and npm for development tools. | `server/mix.exs`; `package.json` | – | | `build_floss_tools` | SHOULD | Met | Elixir, Erlang/OTP, Mix, Node.js and npm are all FLOSS. | – | – | | `test` | MUST | Met | ExUnit tests for the server, Node's built-in test runner for the extension (unit, DOM and background suites) and Playwright end-to-end tests, all in the repository under Apache-2.0. How to run them: CONTRIBUTING.md, Setup; CI runs them on every pull request. | `server/test/`; `test/`; `CONTRIBUTING.md:15-30`; `.github/workflows/ci.yml:75,87,177` | – | -| `test_invocation` | SHOULD | Met | mix test (server) and npm test (extension), the standard commands of each ecosystem. | `package.json` `scripts.test`; `CONTRIBUTING.md:19-30` | – | +| `test_invocation` | SHOULD | Met | npm test (Node's built-in test runner) for the extension and mix test for the server, the standard commands of each ecosystem; npm run e2e runs the Playwright end-to-end tests. https://github.com/ScriptKittyOS/kotiko/blob/main/CONTRIBUTING.md#setup | `package.json` `scripts.test`; `CONTRIBUTING.md:19-30` | – | | `test_most` | SUGGESTED | Met | Measured 2026-10-06: the server tests cover 87.8 % of lines (Elixir's cover tool) and the extension's unit, DOM and background tests 91.4 % of lines and 79.1 % of branches of the extension code they load. | `cd server && mix test --cover`: 1000 tests, 2 properties, 0 failures, 87.78 %; `node --test --experimental-test-coverage --test-coverage-include='extension/**' 'test/{unit,dom,bg}/**/*.test.mjs'`: 1857 tests pass, all files 91.42 % lines, 79.13 % branches | A coverage gate in CI is planned (slice 02 addition) | -| `test_continuous_integration` | SUGGESTED | Met | GitHub Actions runs every test suite, the linters and the audits on every push to main and every pull request; main accepts only pull requests whose eight required checks pass. | `.github/workflows/ci.yml:5-8`; ruleset 24531669 (`required_status_checks`) | – | +| `test_continuous_integration` | SUGGESTED | Met | Every change reaches main through a short-lived pull request, and GitHub Actions runs every test suite, the linters and the audits on each pull request and each push to main; main requires 8 checks to pass. https://github.com/ScriptKittyOS/kotiko/actions/workflows/ci.yml | `.github/workflows/ci.yml:5-8`; ruleset 24531669 (`required_status_checks`) | – | | `test_policy` | MUST | Met | Every pull request that adds or changes behaviour adds or updates automated tests in the same pull request; every bug fix adds a regression test (CONTRIBUTING.md, Tests). | `CONTRIBUTING.md:55-63`; `.github/pull_request_template.md:9-10` | – | | `tests_are_added` | MUST | Met | The most recent major features came with tests, for example backup, restore, export and delete (commit c5144de: server, background, DOM and end-to-end tests), the accessibility baseline (59af37e: axe and screen-reader end-to-end tests) and the docs site (97e2943: build and page tests). | `git show --stat c5144de 59af37e 97e2943` | – | -| `tests_documented_added` | SUGGESTED | Met | Documented in CONTRIBUTING.md, Tests, and in the pull request template's checklist. | `CONTRIBUTING.md:55-63`; `.github/pull_request_template.md:9-10` | – | +| `tests_documented_added` | SUGGESTED | Met | The instructions for change proposals say it: CONTRIBUTING.md, Tests (https://github.com/ScriptKittyOS/kotiko/blob/main/CONTRIBUTING.md#tests), requires tests with every behaviour change and says major new functionality is not merged without them, and the pull request template's checklist asks for them. | `CONTRIBUTING.md:55-63`; `.github/pull_request_template.md:9-10` | – | | `warnings` | MUST | Met | The server compiles with warnings as errors on the oldest and newest supported Elixir; Credo, ESLint (with eslint-plugin-no-unsanitized), web-ext lint and ShellCheck run in CI. | `.github/workflows/ci.yml:69-70,86,149`; `eslint.config.js` | – | | `warnings_fixed` | MUST | Met | CI fails on any compiler warning and any Credo, ESLint or ShellCheck finding. web-ext lint reports one known warning on the source manifest: Firefox ignores the Chrome service-worker key, which is intended (the source lists both forms; the Firefox package is built without it). | `mix compile --warnings-as-errors`: exit 0; `mix credo`: no issues; `npm run lint`: ESLint clean, web-ext 0 errors, 1 warning (`BACKGROUND_SERVICE_WORKER_IGNORED`) (2026-10-06) | – | -| `warnings_strict` | SUGGESTED | Unmet | We compile with warnings as errors and ESLint reports unused disable directives as errors, but Credo doesn't yet run in strict mode and ESLint uses the recommended set. Credo strict mode and stricter ESLint rules are planned. | `.github/workflows/ci.yml:70`; `eslint.config.js:31-34`; `mix credo --strict`: 6 readability issues, 12 design suggestions (2026-10-06) | Slice 02 addition (Credo strict) | +| `warnings_strict` | SUGGESTED | Met | Warnings are as strict as practical, and any warning fails CI on every pull request: the server compiles and runs its tests with --warnings-as-errors; Credo runs in strict mode (server/.credo.exs sets strict: true, so a plain mix credo reports the same); ESLint adds about 60 rules beyond its recommended set (among them eqeqeq, no-shadow, no-unused-vars for every argument, no-param-reassign, no-implicit-coercion, no-loop-func, prefer-const, no-var, no-eval) and treats unused disable directives as errors; web-ext lint, ShellCheck and Sobelow fail on findings too. The few rules not enabled, and why, are written down. https://github.com/ScriptKittyOS/kotiko/blob/main/docs/CODING_STANDARDS.md | `server/.credo.exs`; `.github/workflows/ci.yml` (`mix credo --strict`, `mix test --warnings-as-errors`); `eslint.config.js` (STRICTER block); `docs/CODING_STANDARDS.md` | – | ## Passing: Security @@ -140,7 +139,7 @@ What has to happen for the badge, and to keep it, and who does it: | `crypto_floss` | MUST | Met | Erlang/OTP (crypto and ssl, built on OpenSSL) and the Web Crypto API (implemented by Firefox and Chromium) are FLOSS. | – | – | | `crypto_keylength` | MUST | Met | By default Kotiko's secrets are 256-bit: the server generates the API token from 32 random bytes. A token the person sets themselves must be at least 24 characters, and the server suggests openssl rand -hex 24 (192 bits). Hashes are SHA-256. TLS key exchange and lengths are left to Erlang/OTP's and the browser's defaults; Kotiko offers no setting that weakens them. | `server/lib/kotiko/token.ex:11,48,59`; `server/lib/kotiko/config.ex:297`; `docs/reference/configuration.md:91` | – | | `crypto_working` | MUST | Met | No MD4, MD5, DES, RC4, ECB or disabled certificate checks in Kotiko's code. | `grep -rniE "md5\|md4\|rc4\|:des\b\|verify_none\|\becb\b" server/lib server/config extension scripts --include=*.ex --include=*.exs --include=*.js --include=*.mjs --include=*.sh` finds nothing (2026-10-06) | – | -| `crypto_weaknesses` | SHOULD | Met | No SHA-1 or other weak algorithms or modes in Kotiko's code; SHA-256 is used where a hash is needed. | `grep -rniE "sha1\|sha-1"` over the same paths finds nothing (2026-10-06) | – | +| `crypto_weaknesses` | SHOULD | Met | No default security mechanism depends on a weak algorithm or mode: there is no SHA-1, MD4, MD5, RC4, DES, ECB or disabled certificate check in Kotiko's code. SHA-256 is used where a hash is needed (lookup cache keys, the PKCE challenge), and TLS is left to Erlang/OTP and the browser. | `grep -rniE "sha1\|sha-1"` over the same paths finds nothing (2026-10-06) | – | | `crypto_pfs` | SHOULD | Met | Kotiko negotiates no keys itself. Its TLS connections use Erlang/OTP (TLS 1.3 and 1.2 by default from OTP 26, the oldest supported) and the browser, which use forward-secret key exchange (TLS 1.3 always; ECDHE for TLS 1.2). | `.github/workflows/ci.yml:46-47` (OTP 26.2 and 28.1) | – | | `crypto_password_storage` | MUST | N/A | Kotiko has no user passwords. The server is protected by one random API token (bearer token) and compares it in constant time; there are no user accounts. | `server/lib/kotiko/router.ex:206-214`; `server/lib/kotiko/token.ex` | – | | `crypto_random` | MUST | Met | Tokens, ids, salts and nonces come from cryptographically secure generators: :crypto.strong_rand_bytes on the server, crypto.getRandomValues and crypto.randomUUID in the extension (including the PKCE verifier). Math.random is used only for non-secret values (instance labels, confetti). | `server/lib/kotiko/token.ex:59`; `server/lib/kotiko/uuid7.ex:19`; `extension/lib/pkce.js:34`; `extension/lib/store.js:85`; `extension/content.js:70`; non-secret: `extension/content.js:471`, `extension/lib/word-source.js:53`, `extension/ui/confetti.js:121` | – | @@ -155,12 +154,12 @@ What has to happen for the badge, and to keep it, and who does it: | Criterion | Level | Status | Answer | Evidence | Needs | |---|---|---|---|---|---| | `static_analysis` | MUST [J] [N/A J] | Met | Credo (Elixir), ESLint with eslint-plugin-no-unsanitized and web-ext lint (JavaScript and the extension), ShellCheck (shell) and gitleaks (secrets) run on every push and pull request, so on every release candidate, and main accepts only pull requests that pass them. | `.github/workflows/ci.yml:20-37,70,86,149`; `eslint.config.js`; `server/.credo.exs` | – | -| `static_analysis_common_vulnerabilities` | SUGGESTED | Met | eslint-plugin-no-unsanitized looks for DOM cross-site scripting, and ESLint bans innerHTML, outerHTML and insertAdjacentHTML in the extension; web-ext lint flags unsafe extension patterns; gitleaks looks for leaked secrets. A security analyser for Elixir (Sobelow) is planned. | `eslint.config.js:8-13,44-49`; `.github/workflows/ci.yml:20-37` | Slice 02 addition (Sobelow, CodeQL) | +| `static_analysis_common_vulnerabilities` | SUGGESTED | Met | Two security analysers look for common vulnerabilities on every pull request and push to main, and a finding fails the build: Sobelow for the Elixir server (injection, traversal, unsafe deserialization, atom exhaustion; every finding fails, false positives are skipped per function with a written reason) and CodeQL with the security-extended queries for JavaScript and the GitHub Actions workflows (a required check). For the extension, eslint-plugin-no-unsanitized and a ban on HTML string sinks cover DOM cross-site scripting. The policy says what blocks a merge or a release: https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/dependency-and-static-analysis-policy.md | `server/.sobelow-conf`; `.github/workflows/codeql.yml`; `.github/workflows/ci.yml` (server job, `mix sobelow`); `eslint.config.js`; ruleset 24531669 requires `analyze (actions)`, `analyze (javascript-typescript)` | – | | `static_analysis_fixed` | MUST | Met | No static analysis finding is open: CI fails on any, and pull requests are merged only when CI passes. None so far was an exploitable vulnerability. | `.github/workflows/ci.yml`; ruleset 24531669 | – | | `static_analysis_often` | SUGGESTED | Met | On every push to main and every pull request. | `.github/workflows/ci.yml:5-8` | – | -| `dynamic_analysis` | SUGGESTED | Unmet | Not yet: there is no fuzzer or scanner. The tests run the real server and extension (including end-to-end tests against the real server), and property-based tests (StreamData) vary inputs for two parsers, but that is not yet a dynamic analysis of each release. Planned: property tests for the parsers and matcher, or a ZAP baseline scan. | `server/test/kotiko/word_test.exs`, `server/test/kotiko/text_test.exs` (`property`) | Slice 53 section 8 (gold groundwork) | -| `dynamic_analysis_unsafe` | SUGGESTED | N/A | Kotiko is written in Elixir and JavaScript, which are memory-safe. | – | – | -| `dynamic_analysis_enable_assertions` | SUGGESTED | Unmet | The tests assert heavily, but there is no assertion-enabled build or test mode yet. | – | Slice 53 section 8 | +| `dynamic_analysis` | SUGGESTED | Unmet | Not yet for every release. CI runs property-based tests (StreamData) that feed generated input to two server parsers, the end-to-end tests drive the real extension against the real server, and axe scans every interface, but no fuzzer or scanner is applied to each proposed release yet. Planned: property-based tests for the request parsers, the URL and response validators and the matcher, run in the release workflow, or an OWASP ZAP baseline scan of the server. | `server/test/kotiko/word_test.exs`, `server/test/kotiko/text_test.exs` (`property`) | Slice 53 section 8 (gold groundwork) | +| `dynamic_analysis_unsafe` | SUGGESTED | N/A | Kotiko's code is Elixir and JavaScript, both memory-safe. The only memory-unsafe code is SQLite inside a dependency (exqlite), which the project does not write or build itself. | – | – | +| `dynamic_analysis_enable_assertions` | SUGGESTED | Unmet | Not yet. The tests assert heavily, but the shipped code has no run-time assertions that are switched on during dynamic analysis. Planned with the property-based tests above. | – | Slice 53 section 8 | | `dynamic_analysis_fixed` | MUST | N/A | No dynamic analysis tool is run yet, so none has reported a vulnerability. | – | – | ## Silver and gold @@ -173,12 +172,12 @@ earned, and gold once silver is; the answers can be entered before that. | Criterion | Level | Status | Answer | Evidence | Needs | |---|---|---|---|---|---| -| `achieve_passing` | MUST | Unmet | Not yet. The passing answers are being entered on this entry now; the repository became public on 2026-10-06. Silver is submitted after passing is earned. | bestpractices.dev/projects/15259.json (saved as scratchpad `p.json`): `badge_level` in_progress, `badge_percentage_0` 0, `achieved_passing_at` null | Maintainer fills and saves the passing form from `docs/best-practices.md` | +| `achieve_passing` | MUST | Met | Kotiko earned the passing badge on 2026-10-06: https://www.bestpractices.dev/projects/15259 | bestpractices.dev entry 15259 (passing 100 %) | – | | `contribution_requirements` | MUST [URL] | Met | CONTRIBUTING.md states what an acceptable contribution needs: the rules that protect users (text-only DOM, no runtime npm code in the extension, no user text or secrets in logs, no secrets in the repository, no network in tests), automated tests with every behaviour change and a regression test with every fix, Conventional Commits, SPDX headers, and the coding standards in docs/CODING_STANDARDS.md, which CI checks. https://github.com/ScriptKittyOS/kotiko/blob/main/CONTRIBUTING.md and https://github.com/ScriptKittyOS/kotiko/blob/main/docs/CODING_STANDARDS.md | `CONTRIBUTING.md:38-79`; `docs/CODING_STANDARDS.md:1-101` | – | | `dco` | SHOULD [URL] | Met | Kotiko uses the Developer Certificate of Origin 1.1 (https://developercertificate.org/): every commit must carry a Signed-off-by line for its author, made with git commit -s, as CONTRIBUTING.md requires (https://github.com/ScriptKittyOS/kotiko/blob/main/CONTRIBUTING.md#developer-certificate-of-origin). CI's required secrets job checks every commit of every pull request (scripts/check-dco.mjs); only bots and merge commits are exempt. Contributions are licensed under Apache-2.0 section 5 (inbound = outbound). | `CONTRIBUTING.md` "Developer Certificate of Origin"; `scripts/check-dco.mjs`; `.github/workflows/ci.yml` job `secrets`; `test/unit/repo-checks.test.mjs`; `slices/DECISIONS.md` 2026-10-06 | – | | `governance` | MUST [URL] | Met | Kotiko is maintainer-led. Day-to-day decisions are made in issues and pull requests by lazy consensus (no unresolved maintainer objection after 7 days); the lead maintainer makes the final call on disagreements and records product decisions with who decided and why in slices/DECISIONS.md. Disputes unresolved after 14 days go to the lead maintainer. With three or more maintainers, contested decisions go to a simple majority. GOVERNANCE.md also says how maintainers join and step down and how the document itself changes. https://github.com/ScriptKittyOS/kotiko/blob/main/GOVERNANCE.md | `GOVERNANCE.md:8-34` | – | | `code_of_conduct` | MUST [URL] | Met | Contributor Covenant 2.1, in the standard location: https://github.com/ScriptKittyOS/kotiko/blob/main/CODE_OF_CONDUCT.md . Reports go to security@scriptkittyos.com; GitHub lists it in the repository's community profile. | `CODE_OF_CONDUCT.md:1,39,73`; `gh api repos/ScriptKittyOS/kotiko/community/profile` lists `code_of_conduct_file` | – | -| `roles_responsibilities` | MUST [URL] | Met | GOVERNANCE.md defines each role and the tasks it must perform: lead maintainer, maintainer (reviews and merges, weekly triage with a first response within 7 days, approves releases, enforces the code of conduct), release manager, security response lead, steward, locale reviewer and contributor (https://github.com/ScriptKittyOS/kotiko/blob/main/GOVERNANCE.md#roles). MAINTAINERS.md says who holds which role today: one person holds every role except steward, and the roles not yet filled are listed (https://github.com/ScriptKittyOS/kotiko/blob/main/MAINTAINERS.md). The kotiko-maintainers team, which owns every file through .github/CODEOWNERS, holds exactly the active maintainers. | `GOVERNANCE.md:36-49`; `MAINTAINERS.md:12-24`; `.github/CODEOWNERS:6`; `gh api orgs/ScriptKittyOS/teams/kotiko-maintainers/members` gives HackTuah only | – | +| `roles_responsibilities` | MUST [URL] | Met | GOVERNANCE.md defines each role and the tasks it must perform: lead maintainer, maintainer (reviews and merges, weekly triage with a first response within 7 days, approves releases, enforces the code of conduct), release manager, security response lead, an optional steward, locale reviewer and contributor (https://github.com/ScriptKittyOS/kotiko/blob/main/GOVERNANCE.md#roles). MAINTAINERS.md says who holds which role today: the lead maintainer, Ayla Croft (@HackTuah), holds the lead, release manager and security response roles, and @minitru is a maintainer; the maintainers are exactly the repository's collaborators with the Maintain role, and .github/CODEOWNERS names them (https://github.com/ScriptKittyOS/kotiko/blob/main/MAINTAINERS.md). | `GOVERNANCE.md` "Roles"; `MAINTAINERS.md`; `.github/CODEOWNERS`; `gh api repos/ScriptKittyOS/kotiko/collaborators` | – | | `access_continuity` | MUST [URL] | Met | GOVERNANCE.md has a Continuity section: if any one maintainer becomes unavailable the other holds the access to triage and close issues, review and merge pull requests, and publish a release (a release tag signed with their own key); nothing in the ordinary workflow requires a specific individual. Kotiko has two maintainers, both repository collaborators, and the organization enforces two-factor authentication for everyone with access. The exceptions (organization administration, the domains, the security mailbox, the store publisher accounts) are named, and the continuity plan lists how each is handed over. https://github.com/ScriptKittyOS/kotiko/blob/main/GOVERNANCE.md#continuity | `GOVERNANCE.md` "Continuity"; `MAINTAINERS.md` (two active maintainers); `docs/governance/continuity.md`; `gh api repos/ScriptKittyOS/kotiko/collaborators`: HackTuah admin, minitru maintain; `gh api orgs/ScriptKittyOS`: two_factor_requirement_enabled true | – | | `bus_factor` | SHOULD [URL] | Unmet | Kotiko has two maintainers since 2026-10-06, but every commit so far was written by the lead maintainer, so the bus factor is still 1 in practice. We are working on it: the architecture, the assurance case and the coding standards are written down, every planned piece of work in slices/ is a buildable spec, and GOVERNANCE.md says how the maintainers continue without any one of them. https://github.com/ScriptKittyOS/kotiko/blob/main/docs/governance/continuity.md#bus-factor | `git shortlog -sne HEAD`: one author; `MAINTAINERS.md`; `GOVERNANCE.md` "Continuity" | The second maintainer contributing code and reviews regularly | | `documentation_roadmap` | MUST [URL] | Met | ROADMAP.md covers the year through October 2027: what comes now (the first public release, in four phases), in the six months after it, and later, plus what Kotiko will not do (no paid tier or hosted accounts, no bundled word packs, no telemetry, no bundler in the extension). The lead maintainer reviews it every quarter and at each minor release. https://github.com/ScriptKittyOS/kotiko/blob/main/ROADMAP.md | `ROADMAP.md:3,9-90` | – | @@ -210,13 +209,13 @@ earned, and gold once silver is; the answers can be entered before that. | Criterion | Level | Status | Answer | Evidence | Needs | |---|---|---|---|---|---| | `coding_standards` | MUST [URL] [N/A J] | Met | docs/CODING_STANDARDS.md names the style guide for each language: Elixir follows mix format and Credo, then the community Elixir Style Guide; JavaScript follows ESLint and the project's own style section; shell follows ShellCheck and the Google Shell Style Guide; HTML and CSS use the design tokens with no inline scripts. CONTRIBUTING.md requires contributions to follow it. https://github.com/ScriptKittyOS/kotiko/blob/main/docs/CODING_STANDARDS.md | `docs/CODING_STANDARDS.md:17-86`; `CONTRIBUTING.md:67-68` | – | -| `coding_standards_enforced` | MUST [J] [N/A J] | Unmet | Partly. CI enforces mix format, Credo and compiler warnings for Elixir, ESLint and web-ext lint for JavaScript, ShellCheck for shell, and REUSE headers on every pull request. The JavaScript layout rules (double quotes, semicolons, two-space indent) are not yet checked by a formatter; adding Prettier (or ESLint stylistic rules) to CI is planned. | `.github/workflows/ci.yml:66-75,85-88,141,147`; `docs/CODING_STANDARDS.md:43-47`; `package.json:12` (lint = eslint + web-ext lint, no formatter) | Agent: add a Prettier check (one mechanical reformat commit, `printWidth: 100`) or @stylistic ESLint rules to CI. Maintainer: decide slice 53 open question 4 | +| `coding_standards_enforced` | MUST [J] [N/A J] | Met | CI enforces the coding standards on every pull request: mix format --check-formatted and Credo in strict mode for Elixir; for JavaScript, ESLint with @stylistic/eslint-plugin layout rules that match the code as written (indentation, quotes, semicolons, trailing commas, spacing) plus the stricter rules; ShellCheck for shell scripts. npm run lint:fix and mix format apply the layout. Prettier was measured and rejected because reformatting would push the extension popup past its size budget; the numbers are in the standards document. https://github.com/ScriptKittyOS/kotiko/blob/main/docs/CODING_STANDARDS.md | `eslint.config.js` (LAYOUT and STRICTER blocks); `.github/workflows/ci.yml`; `docs/CODING_STANDARDS.md`; `package.json` `lint`, `lint:fix` | – | | `build_standard_variables` | MUST [J] [N/A J] | N/A | The project builds no native binaries. The server is Elixir compiled to BEAM bytecode by Mix, and the extension is JavaScript packed into a zip as is. The only native code (SQLite, in the exqlite dependency) is built by that dependency, not by the project. | `server/mix.exs:32-46`; `scripts/build-extension.mjs:1-20` | – | | `build_preserve_debug` | SHOULD [J] [N/A J] | Met | Mix compiles the server with debug information (Elixir's default) and nothing strips it. The extension is shipped as source: the build copies the files unchanged into the zip, with no minification or bundling. | `server/mix.exs:7-20` (no debug_info override); `docs/verify.md:61-68` | – | | `build_non_recursive` | MUST [J] [N/A J] | Met | One Mix project builds the server (server/mix.exs) and one script packs the extension (scripts/build-extension.mjs); neither recurses into subdirectories with cross-dependencies. | `server/mix.exs`; `scripts/build-extension.mjs` | – | | `build_repeatable` | MUST [J] [N/A J] | Met | Yes. scripts/build-extension.mjs builds both extension zips from the commit with entries sorted by path, the commit's timestamp, fixed permissions and a pinned pure-JavaScript deflate, so the bytes don't depend on the machine or time zone. The release workflow builds twice and fails if the zips differ, and building twice on 2026-10-06 gave identical SHA-256 hashes. How to check it yourself: https://github.com/ScriptKittyOS/kotiko/blob/main/docs/reproducible-builds.md | `.github/workflows/release.yml:111-118`; `docs/reproducible-builds.md:29-50`; ran `node scripts/build-extension.mjs --version 0.2.0 --skip-lint` twice into scratch dirs: chrome zip c7fceffd...38c1 and firefox zip ebee58fd...4ab0 both times, `cmp` identical | – | -| `installation_common` | MUST [J] [N/A J] | Unmet | Not yet. The browser stores are the convention for installing and removing an extension, and Kotiko is not listed there until its first release. Today it is loaded from the repository's extension folder and removed from the browser's extensions page. The server runs from a git checkout (run.sh, or a systemd user service from server/install-service.sh), which has no uninstall step. Store listings come with the first release; a Docker image, a release tarball and uninstall steps are planned. | `README.md:54-64`; `site/src/content/docs/install.mdx`; `grep -n uninstall server/install-service.sh server/run.sh` finds nothing; `gh release list` empty | Maintainer: publish v1.0.0 to the Chrome Web Store and Firefox Add-ons. Agent: `install-service.sh --uninstall` and its docs (slice 40 addition) | -| `installation_standard_variables` | MUST [J] [N/A J] | Unmet | The extension is installed by the browser and the server runs from its own folder, so there is no install prefix or DESTDIR. The one installer, server/install-service.sh, writes its systemd user unit to ~/.config/systemd/user without honouring XDG_CONFIG_HOME, and the default data folder ignores XDG_DATA_HOME; both are to be fixed. | `server/install-service.sh:12-13,58`; `server/lib/kotiko/data_dir.ex:41` | Agent: honour XDG_CONFIG_HOME in install-service.sh and XDG_DATA_HOME for the default data folder (keeping the old default readable). Alternative if the maintainer prefers: answer N/A (no installation system with a standard convention) | +| `installation_common` | MUST [J] [N/A J] | Met | The server installs and uninstalls with one script: server/install-service.sh sets it up as a systemd user service and install-service.sh --uninstall removes exactly what it added, keeping the learner's words unless --delete-data is given. The extension is installed and removed through the browser's own extension manager (from source with Load unpacked today; from the Chrome Web Store and Firefox Add-ons from v1.0.0). https://github.com/ScriptKittyOS/kotiko/blob/main/site/src/content/docs/server/service.md | `server/install-service.sh` (`--uninstall`, `--delete-data`, `--help`); `server/test/kotiko/scripts_test.exs`; `site/src/content/docs/server/service.md` "Remove the service" | – | +| `installation_standard_variables` | MUST [J] [N/A J] | Met | The server follows the XDG Base Directory convention: its default data folder is $XDG_DATA_HOME/kotiko (falling back to ~/.local/share/kotiko, and keeping an existing folder there), and install-service.sh writes its systemd user unit to $XDG_CONFIG_HOME/systemd/user. It installs nothing to system locations, so DESTDIR and PREFIX don't apply, as the script says. The extension is installed by the browser. https://github.com/ScriptKittyOS/kotiko/blob/main/docs/reference/configuration.md | `server/lib/kotiko/data_dir.ex`; `server/install-service.sh`; `server/test/kotiko/data_dir_test.exs`; `server/test/kotiko/scripts_test.exs` | – | | `installation_development_quick` | MUST [J] [N/A J] | Met | CONTRIBUTING.md Setup installs everything needed to change and test Kotiko with each ecosystem's standard tools: mix deps.get and mix test for the server, npm ci and npm test for the extension, and npx playwright install chromium for the end-to-end tests. https://github.com/ScriptKittyOS/kotiko/blob/main/CONTRIBUTING.md#setup | `CONTRIBUTING.md:15-36` | – | | `external_dependencies` | MUST [URL] [J] [N/A J] | Met | Dependencies are declared and locked in each ecosystem's standard files: server/mix.exs with server/mix.lock (Hex), package.json with package-lock.json (npm, development tools only; the extension ships no npm code), and site/package.json with its lock file for the docs site. External services are listed in docs/ARCHITECTURE.md section 6, and the release workflow attaches a CycloneDX SBOM of the server to each release. https://github.com/ScriptKittyOS/kotiko/blob/main/server/mix.exs | `server/mix.exs:32-46`; `package.json:15-33`; `site/package-lock.json`; `docs/ARCHITECTURE.md:246-282`; `.github/workflows/release.yml:119-127` | – | | `dependency_monitoring` | MUST [J] [N/A J] | Met | Dependabot alerts and security updates are on, with weekly version updates for Mix, npm and GitHub Actions, and CI runs mix hex.audit and mix deps.audit on every pull request. The open alerts on 2026-10-06 are all in development-only npm tools, never shipped: shell-quote and postcss-selector-parser have fixes in an open pull request, and node-forge (pulled in by web-ext's Android debugging library, no patched version yet) is reached only when web-ext debugs on Android over adb, which Kotiko's scripts never do (they run web-ext lint and sign), so it is not exploitable here. | `gh api repos/ScriptKittyOS/kotiko/dependabot/alerts?state=open`: #1 node-forge (high, development, no patch), #2 shell-quote (critical, development, fixed 1.11.0), #3 postcss-selector-parser (medium, development, site/package-lock.json); package-lock: web-ext -> fx-runner -> shell-quote, web-ext -> @devicefarmer/adbkit -> node-forge; `gh api repos/ScriptKittyOS/kotiko/vulnerability-alerts` 204; `security_and_analysis.dependabot_security_updates: enabled`; `.github/dependabot.yml:1-23`; `.github/workflows/ci.yml:72-73`; `package.json:12`; `.github/workflows/release.yml:351-353` | Maintainer: merge PR #9. Agent: add `/site` to dependabot.yml (version updates miss it today) and an npm audit or OSV-Scanner step to CI | @@ -227,7 +226,7 @@ earned, and gold once silver is; the answers can be entered before that. | `test_statement_coverage80` | MUST [J] [N/A J] | Met | Measured 2026-10-06 with FLOSS tools: the server's ExUnit suite (1000 tests and 2 properties) covers 87.8 % of lines (mix test --cover, Erlang's cover tool), and the extension's Node test suites cover 92.6 % of the lines of the extension code they load (node --test --experimental-test-coverage); three small extension files (120 lines) are not loaded by them. The end-to-end tests come on top. A coverage gate in CI is planned. | Scratch copy of HEAD: `MIX_ENV=test mix test --cover` gives 1000 tests, 2 properties, 0 failures, Total 87.78 %; `node --test --experimental-test-coverage --test-coverage-include='extension/**' 'test/{unit,dom,bg}/**/*.test.mjs'` gives all files 92.59 % lines, 79.13 % branches (1856 of 1857 pass; the one failure needs a git commit, absent in the scratch copy); not loaded: `extension/content/connect.js`, `extension/privacy.js`, `extension/ui/theme.js` | Agent (optional): CI coverage gate so the number can't slip | | `test_policy_mandated` | MUST [J] [N/A J] | Met | CONTRIBUTING.md, Tests: every pull request that adds or changes behaviour adds or updates automated tests in the same pull request, and major new functionality is not merged without tests. https://github.com/ScriptKittyOS/kotiko/blob/main/CONTRIBUTING.md#tests | `CONTRIBUTING.md:55-63` | – | | `tests_documented_added` | MUST [J] [N/A J] | Met | The instructions for change proposals say it: CONTRIBUTING.md, Tests (https://github.com/ScriptKittyOS/kotiko/blob/main/CONTRIBUTING.md#tests), requires tests with every behaviour change and says major new functionality is not merged without them, and the pull request template's checklist asks for them. | `CONTRIBUTING.md:55-63`; `.github/pull_request_template.md:9-10` | – | -| `warnings_strict` | MUST [J] [N/A J] | Unmet | Partly. The server compiles with --warnings-as-errors on two Elixir versions, and CI fails on any Credo, ESLint (recommended rules plus no-unsanitized, unused disable directives as errors), web-ext lint or ShellCheck finding. Not yet maximal: Credo runs in its default mode rather than --strict (which reports 18 more readability and design issues today), and ESLint uses only the recommended rule set. Strict Credo and stricter ESLint rules are planned. | `.github/workflows/ci.yml:69-70`; `eslint.config.js:31-34`; `mix credo --strict` (scratch copy): 6 code readability issues, 12 software design suggestions | Agent: fix the 18 findings, run `mix credo --strict` in CI, add stricter ESLint rules | +| `warnings_strict` | MUST [J] [N/A J] | Met | Warnings are as strict as practical, and any warning fails CI on every pull request: the server compiles and runs its tests with --warnings-as-errors; Credo runs in strict mode (server/.credo.exs sets strict: true, so a plain mix credo reports the same); ESLint adds about 60 rules beyond its recommended set (among them eqeqeq, no-shadow, no-unused-vars for every argument, no-param-reassign, no-implicit-coercion, no-loop-func, prefer-const, no-var, no-eval) and treats unused disable directives as errors; web-ext lint, ShellCheck and Sobelow fail on findings too. The few rules not enabled, and why, are written down. https://github.com/ScriptKittyOS/kotiko/blob/main/docs/CODING_STANDARDS.md | `server/.credo.exs`; `.github/workflows/ci.yml` (`mix credo --strict`, `mix test --warnings-as-errors`); `eslint.config.js` (STRICTER block); `docs/CODING_STANDARDS.md` | – | ## Silver: Security @@ -236,7 +235,7 @@ earned, and gold once silver is; the answers can be entered before that. | `implement_secure_design` | MUST [J] [N/A J] | Met | The assurance case shows how each Saltzer and Schroeder principle is applied, with code and tests: deny by default (one authorization plug before routing, only /health open, constant-time token comparison), loopback bind by default, a Host allowlist against DNS rebinding, complete mediation tested over encoded paths and through a real socket, one message router in the extension that checks every sender, secrets kept where content scripts can't read them, and a small attack surface. Where a principle is applied only weakly (one token grants everything on the server), it says so. https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/assurance-case.md#4-secure-design-principles | `docs/security/assurance-case.md:102-117`; `server/lib/kotiko/router.ex:11,206-214`; `server/lib/kotiko/config.ex:19` | – | | `crypto_weaknesses` | MUST [J] | Met | No default security mechanism depends on a weak algorithm or mode: there is no SHA-1, MD4, MD5, RC4, DES, ECB or disabled certificate check in Kotiko's code. SHA-256 is used where a hash is needed (lookup cache keys, the PKCE challenge), and TLS is left to Erlang/OTP and the browser. | `grep -rniE` for verify_none, sha1, sha-1, md5, rc4, :des, ecb, insecure, rejectUnauthorized over `server/lib server/config extension` (*.ex, *.exs, *.js): no match (2026-10-06); `server/lib/kotiko/llm/cache.ex:26,47`; `extension/lib/pkce.js:36` | – | | `crypto_algorithm_agility` | SHOULD [J] | Met | Kotiko fixes no cipher itself: TLS versions and cipher suites are negotiated by Erlang/OTP and the browser, which support several (AES-GCM, ChaCha20-Poly1305) and change with their updates. The only algorithm Kotiko names is SHA-256, for internal cache keys and for the PKCE S256 method that the OAuth standard defines. | `grep -rn` for versions:, :tlsv, ssl: in `server/lib server/config`: no match; `server/lib/kotiko/llm/cache.ex:26,47`; `extension/lib/pkce.js:36` | – | -| `crypto_credential_agility` | MUST [J] | Unmet | Partly. The server's API token lives in its own file (api-token in the data folder, mode 0600) and is replaced with mix kotiko.token --rotate, without recompiling. The model API key, the Telegram bot token and the transcription key are read from environment variables, usually set in server/.env next to other settings; reading each from its own file (for example LLM_API_KEY_FILE) is planned. In the extension, keys are kept in their own secrets store, separate from settings, and replaced in Settings. | `docs/reference/configuration.md:83-104,127-133,226`; `docs/security/assurance-case.md:39`; `extension/lib/store.js:13,62`; `grep -rn _FILE server/lib server/config`: no match | Agent: `*_FILE` variants for every server secret, documented in configuration.md (slice 40 addition) | +| `crypto_credential_agility` | MUST [J] | Met | Credentials are kept apart from the code and the other settings, and each can be replaced without changing either: every server secret (API_TOKEN, LLM_API_KEY, TELEGRAM_BOT_TOKEN, TRANSCRIBE_API_KEY) can be given as NAME_FILE=/path, the Docker secrets and systemd credentials convention, so a key is replaced by replacing a file and restarting. The API token can also be rotated with mix kotiko.token --rotate. Documented at https://github.com/ScriptKittyOS/kotiko/blob/main/docs/reference/configuration.md | `server/lib/kotiko/config.ex` `read_secret_files/2`; `server/test/kotiko/config_test.exs`; `docs/reference/configuration.md` "Secrets in files" | – | | `crypto_used_network` | SHOULD [J] | Met | Kotiko's own outbound traffic uses HTTPS by default: model providers, OpenRouter, Telegram and Wiktionary. The server listens on 127.0.0.1 by default, so its plain-HTTP API doesn't leave the computer unless the owner sets BIND to another address; then the server logs a warning at start (daily for a public address), and the extension warns under the address field when the token would cross a network unencrypted. A plain-HTTP model endpoint (such as a local Ollama) is used only when the user enters one. | `server/lib/kotiko/config.ex:19`; `docs/security/assurance-case.md:29`; `extension/lib/url.js:85-99`; `docs/reference/configuration.md` (LLM_URL default `https://openrouter.ai/api/v1`) | – | | `crypto_tls12` | SHOULD [J] | Met | TLS comes from Erlang/OTP (26 and newer, which offer TLS 1.2 and 1.3 by default) for the server's outbound requests and from the browser for the extension. Kotiko never sets or lowers the TLS version. | `.github/workflows/ci.yml:46-47` (OTP 26.2 and 28.1); `grep -rn` for versions:, :tlsv, ssl: in `server/lib server/config`: no match | – | | `crypto_certificate_verification` | MUST [J] | Met | The server makes HTTPS requests with Req (Mint over Erlang/OTP ssl), which verifies the certificate chain and host name by default; Kotiko's code sets no verify option anywhere. The extension's requests go through the browser, which always verifies certificates. | `grep -rniE` for verify_none and rejectUnauthorized over `server/lib server/config extension`: no match; `server/lib/kotiko/llm/client.ex:41` (only a connect timeout is set) | – | @@ -244,14 +243,14 @@ earned, and gold once silver is; the answers can be entered before that. | `signed_releases` | MUST [J] [N/A J] | N/A | No release has been published yet, so there is nothing to sign today. The pipeline for the first one (v1.0.0) is in place and signs everything it publishes: the release manager signs the tag with their own SSH or GPG key; the workflow refuses a tag not signed by a key listed in .github/allowed_signers on main, builds the zips reproducibly, and publishes them with SHA256SUMS, a CycloneDX SBOM and Sigstore build-provenance attestations (keyless, so no signing key is stored on GitHub). How users verify: https://github.com/ScriptKittyOS/kotiko/blob/main/docs/verify.md | `.github/workflows/release.yml:2-17,50-75,140-165`; `.github/allowed_signers:13-14` (no key yet); `git tag -l` empty; `gh release list` empty; `docs/verify.md:11-52` | Answer Met with the first release (v1.0.0) | | `version_tags_signed` | SUGGESTED [J] | Unmet | No version tag exists yet. From v1.0.0 on, every release tag is signed by the release manager and verified by the release workflow against .github/allowed_signers before anything is built. | `git tag -l` empty; `.github/workflows/release.yml:50-75` | Maintainer: as for signed_releases | | `input_validation` | MUST [J] [N/A J] | Met | Every input from an untrusted source is checked against what is allowed, and refused otherwise: HTTP requests (Host allowlist, exactly one Bearer token, JSON only within size caps, then per-field rules such as text length, canonical language tags, UUID request ids, statuses from a fixed list and limits in range); model answers (the shared word spec: required fields, canonical tags, a script check, length and count caps); server answers in the extension; backup files (format, version, every word re-checked); the server address the user types; and every setting at start. The full table: https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/assurance-case.md#6-inputs-and-how-each-is-validated | `docs/security/assurance-case.md:141-157`; `server/lib/kotiko/router.ex:20-21`; `extension/lib/backup.js:22-29,88-178`; `extension/lib/url.js:52-82` | – | -| `hardening` | SHOULD [J] [N/A J] | Met | Extension pages run under an explicit content security policy (script-src 'self'; object-src 'self') with no inline scripts, and ESLint forbids HTML string sinks (innerHTML, outerHTML, insertAdjacentHTML) so a bug can't turn word data into markup. The server checks the Host header against an allowlist before anything else (against DNS rebinding), caps request bodies, sends no CORS headers, marks answers no-store, binds to loopback by default, compares tokens in constant time, and filters secrets out of logs. Planned: X-Content-Type-Options and similar headers on server responses, and a sandboxed systemd unit. | `extension/manifest.json:46`; `eslint.config.js:10-13,45-49`; `server/lib/kotiko/router.ex:11,20-21,33,214`; `docs/security/assurance-case.md:30,34` | – | +| `hardening` | SHOULD [J] [N/A J] | Met | Extension pages run under an explicit content security policy (script-src 'self'; object-src 'self') with no inline scripts, and ESLint forbids HTML string sinks (innerHTML, outerHTML, insertAdjacentHTML) so a bug can't turn word data into markup. The server checks the Host header against an allowlist before anything else (against DNS rebinding), caps request bodies, sends no CORS headers, marks answers no-store, binds to loopback by default, compares tokens in constant time, and filters secrets out of logs. Planned: X-Content-Type-Options and similar headers on server responses, and a sandboxed systemd unit. Evidence: https://github.com/ScriptKittyOS/kotiko/blob/main/extension/manifest.json and https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/assurance-case.md | `extension/manifest.json:46`; `eslint.config.js:10-13,45-49`; `server/lib/kotiko/router.ex:11,20-21,33,214`; `docs/security/assurance-case.md:30,34` | – | | `assurance_case` | MUST [URL] | Met | docs/security/assurance-case.md: the top claim and its sub-claims mapped to arguments and evidence, the threat model (assets, attackers and what each can reach), the trust boundaries (diagram and table), how each secure design principle is applied, the CWE Top 25 and OWASP Top 10 weaknesses that apply and how each is countered, every input and how it is validated, and the residual risks. https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/assurance-case.md | `docs/security/assurance-case.md:14-41` (claims), `43-66` (threat model), `68-100` (trust boundaries), `102-117` (principles), `119-139` (weaknesses), `141-157` (inputs), `159-183` (residual risks) | Agent: fix the stale lines listed under documentation_current | ## Silver: Analysis | Criterion | Level | Status | Answer | Evidence | Needs | |---|---|---|---|---|---| -| `static_analysis_common_vulnerabilities` | MUST [J] [N/A J] | Unmet | Partly. For JavaScript, ESLint with eslint-plugin-no-unsanitized looks for DOM cross-site scripting, the configuration bans HTML string sinks, and web-ext lint flags unsafe extension patterns; both run on every pull request. For Elixir (the server) no security-focused analyser runs yet; Sobelow, and CodeQL for the JavaScript, are planned. Credo checks code quality, not vulnerabilities. | `eslint.config.js:6,44-49`; `.github/workflows/ci.yml:70,86`; `gh api repos/ScriptKittyOS/kotiko/code-scanning/default-setup` gives `state: not-configured`; `server/mix.exs:32-46` (no sobelow) | Agent: add Sobelow (dev dependency + CI step, failing on findings). Maintainer (setting) or agent (workflow file): CodeQL for JavaScript and Actions | +| `static_analysis_common_vulnerabilities` | MUST [J] [N/A J] | Met | Two security analysers look for common vulnerabilities on every pull request and push to main, and a finding fails the build: Sobelow for the Elixir server (injection, traversal, unsafe deserialization, atom exhaustion; every finding fails, false positives are skipped per function with a written reason) and CodeQL with the security-extended queries for JavaScript and the GitHub Actions workflows (a required check). For the extension, eslint-plugin-no-unsanitized and a ban on HTML string sinks cover DOM cross-site scripting. The policy says what blocks a merge or a release: https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/dependency-and-static-analysis-policy.md | `server/.sobelow-conf`; `.github/workflows/codeql.yml`; `.github/workflows/ci.yml`; ruleset 24531669 required checks; `docs/security/dependency-and-static-analysis-policy.md` | – | | `dynamic_analysis_unsafe` | MUST [J] [N/A J] | N/A | Kotiko's code is Elixir and JavaScript, both memory-safe. The only memory-unsafe code is SQLite inside a dependency (exqlite), which the project does not write or build itself. | `server/mix.exs:32-46`; `ls extension` (JavaScript, HTML, CSS only) | – | ## Gold: Basics @@ -315,7 +314,7 @@ No public release exists yet (git tag is empty; gh release list is empty). The C | Criterion | Level | Status | Answer | Evidence | Needs | |---|---|---|---|---|---| | `osps_ac_01_01` | MUST [N/A J] | Met | The repository is owned by the ScriptKittyOS GitHub organization, which requires two-factor authentication for every member and outside collaborator (organization setting "Require two-factor authentication"). GOVERNANCE.md also asks maintainers to use a passkey, security key or authenticator app, not SMS. https://github.com/ScriptKittyOS/kotiko/blob/main/GOVERNANCE.md#accounts | gh api orgs/ScriptKittyOS: two_factor_requirement_enabled true; GOVERNANCE.md:51-54 | – | -| `osps_ac_02_01` | MUST [N/A J] | Met | GitHub requires a role to be chosen explicitly when a collaborator is invited, and the organization's base permission for members is read only. Write and admin access come only through the kotiko-maintainers team, whose membership changes by pull request to MAINTAINERS.md. https://github.com/ScriptKittyOS/kotiko/blob/main/MAINTAINERS.md | gh api orgs/ScriptKittyOS: default_repository_permission "read"; gh api repos/ScriptKittyOS/kotiko/collaborators: only HackTuah (admin); repos/.../teams: kotiko-maintainers (admin); MAINTAINERS.md:5-10 | – | +| `osps_ac_02_01` | MUST [N/A J] | Met | GitHub requires a role to be chosen explicitly when a collaborator is invited, and the organization's base permission for members is read only. Write access is given only to the maintainers, as repository collaborators with the Maintain role, recorded by a pull request to MAINTAINERS.md. https://github.com/ScriptKittyOS/kotiko/blob/main/MAINTAINERS.md | gh api orgs/ScriptKittyOS: default_repository_permission "read"; gh api repos/ScriptKittyOS/kotiko/collaborators: HackTuah (admin), minitru (maintain); MAINTAINERS.md | – | | `osps_ac_03_01` | MUST [N/A J] | Met | A repository ruleset on the default branch (main) is active with no bypass actors: changes land only through pull requests whose 8 required CI checks pass; direct pushes and force pushes are rejected. | gh api repos/ScriptKittyOS/kotiko/rulesets/24531669: enforcement active, bypass_actors [], rules deletion, non_fast_forward, pull_request, required_status_checks (8) | – | | `osps_ac_03_02` | MUST [N/A J] | Met | The same ruleset on main has the "Restrict deletions" rule with no bypass actors, so the primary branch cannot be deleted without first changing the ruleset, an explicit admin action. | gh api repos/ScriptKittyOS/kotiko/rulesets/24531669: rule type deletion, bypass_actors [] | – | | `osps_br_01_01` | MUST [N/A J] | Met | No workflow expands untrusted metadata (branch names, pull request titles or bodies, commit messages, author names) into a script. There is no pull_request_target, workflow_run or issue_comment trigger. The only ref used in a script is the release tag name, passed through an environment variable and checked against the pattern vX.Y.Z or vX.Y.Z-rc.N before use; release tags can only be created by maintainers (tag ruleset). | grep of .github/workflows for expressions: only github.ref in concurrency groups and github.ref_name via env at release.yml:68,74,194; scripts/verify-tag.sh:48-50 (regex check); grep for pull_request_target, workflow_run, issue_comment finds nothing | – | @@ -344,7 +343,7 @@ No public release exists yet (git tag is empty; gh release list is empty). The C | Criterion | Level | Status | Answer | Evidence | Needs | |---|---|---|---|---|---| | `osps_ac_04_01` | MUST [N/A J] | Met | The repository's and organization's default GITHUB_TOKEN permission is read-only, and every workflow also declares top-level permissions (contents: read, or none). | gh api repos/ScriptKittyOS/kotiko/actions/permissions/workflow: default_workflow_permissions "read"; same for orgs/ScriptKittyOS; ci.yml:10-11; release.yml:26; store-status.yml:15; site.yml:46-47; eval.yml:25-26 | – | -| `osps_br_02_01` | MUST [N/A J] | Met | Every release gets a unique Semantic Versioning identifier (vX.Y.Z, or vX.Y.Z-rc.N for candidates). The release workflow refuses a tag that is not in that form, not signed by a listed maintainer key, not on main, or whose version differs from the extension manifest, server/mix.exs and the release manifest; a tag ruleset stops anyone but maintainers from creating, moving or deleting release tags. No release has been published yet; the first is v1.0.0. | scripts/verify-tag.sh:5-16, 48-80; .github/workflows/release.yml:50-75; scripts/check-versions.mjs; ruleset 24531685 (creation, update, deletion restricted, bypass team kotiko-maintainers) | – | +| `osps_br_02_01` | MUST [N/A J] | Met | Every release gets a unique Semantic Versioning identifier (vX.Y.Z, or vX.Y.Z-rc.N for candidates). The release workflow refuses a tag that is not in that form, not signed by a listed maintainer key, not on main, or whose version differs from the extension manifest, server/mix.exs and the release manifest; only the maintainers can push tags, and a tag ruleset makes every tag permanent (no moving or deleting). No release has been published yet; the first is v1.0.0. | scripts/verify-tag.sh:5-16, 48-80; .github/workflows/release.yml:50-75; scripts/check-versions.mjs; ruleset 24531685 "tags are permanent" (update, deletion and force push blocked, no bypass) | – | | `osps_br_04_01` | MUST [N/A J] | Met | Each release's notes are its CHANGELOG.md section, written for users, plus install and verification steps; the release job fails if the version has no CHANGELOG section, and tagging is refused while entries remain under Unreleased. Security fixes are listed in a Security part of the notes with advisory, CVE, severity and credit. https://github.com/ScriptKittyOS/kotiko/blob/main/CHANGELOG.md | scripts/release-notes.mjs:52-75, 116-129; release.yml:185-200; SECURITY.md:37-38; docs/stores.md:185-197 (checklist); CHANGELOG.md:209-215 | – | | `osps_br_05_01` | MUST [N/A J] | Met | Builds take dependencies only through each ecosystem's standard tools with committed lockfiles: mix deps.get with server/mix.lock, and npm ci with package-lock.json (npm ci --ignore-scripts in jobs that hold store credentials). GitHub Actions are pinned by commit SHA. | .github/workflows/release.yml:107, 125, 269, 331; ci.yml:66, 85; server/mix.lock; package-lock.json | – | | `osps_br_06_01` | MUST [N/A J] | Met | Each release carries SHA256SUMS listing the SHA-256 of every file, and every file including SHA256SUMS gets a Sigstore-signed SLSA build provenance attestation (actions/attest); the GitHub release is created only after attestation succeeds. Release tags must be signed by a maintainer key. No release has been published yet. | .github/workflows/release.yml:128-132, 140-165, 167-168; docs/verify.md:31-49 | The release manager's key must be added to .github/allowed_signers (empty today) before any release can be tagged (maintainer) | @@ -383,8 +382,8 @@ No public release exists yet (git tag is empty; gh release list is empty). The C | `osps_qa_07_01` | MUST [N/A J] | Unmet | Kotiko has one maintainer, so the main ruleset requires a pull request with passing CI but zero approvals; a non-author approval cannot be required until there is a second maintainer. Required review will be turned on when a second maintainer joins. | gh api repos/ScriptKittyOS/kotiko/rulesets/24531669: required_approving_review_count 0; MAINTAINERS.md:12-16; docs/PUBLIC_CHECKLIST.md:15-16 | A second maintainer, then set required approvals to 1 in the main ruleset (maintainer) | | `osps_sa_03_02` | MUST [N/A J] | Met | docs/security/assurance-case.md sections 2 and 3 are the threat model and attack surface analysis: the assets, each attacker (malicious page, injected script, network attacker, bad model answer, unlisted Telegram user, local user, compromised dependency or Action, malicious pull request), what each can reach, every trust boundary with its control and evidence, and the residual risks. https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/assurance-case.md#2-threat-model | docs/security/assurance-case.md:43-101, 159-184 | – | | `osps_vm_04_02` | MUST [N/A J] | Unmet | Kotiko does not yet publish a VEX document. Known example: node-forge (a development-only dependency of web-ext, with no patched version) does not affect the released software, but this is not yet recorded in VEX form. | No VEX file in the repository (no *.vex*, openvex or vex/ found); gh api .../dependabot/alerts?state=open: node-forge (high, no fix, development), shell-quote (critical, development), postcss-selector-parser (medium, site, development) | Add an OpenVEX document (for example vex/kotiko.openvex.json) with node-forge and any other not-affected findings, a policy line on keeping it current, and attach it to releases (agent) | -| `osps_vm_05_01` | MUST [N/A J] | Unmet | SECURITY.md sets fix targets for vulnerabilities in Kotiko and says dependency problems are reported upstream and pinned or patched, but there is no written threshold for remediating SCA findings on dependencies (by severity and runtime versus development scope) or for license findings. | SECURITY.md:33-34, 45-46; docs/security/assurance-case.md:139 | Write a dependency policy (for example docs/security/dependencies.md): severity thresholds and deadlines, runtime vs development scope, exceptions with reason and expiry, allowed licenses (agent) | -| `osps_vm_05_02` | MUST [N/A J] | Unmet | There is no written policy to resolve SCA findings before a release, and the release workflow runs no SCA check (mix deps.audit and hex.audit run on pull requests only; npm dependencies are not audited). | .github/workflows/release.yml:99-138 (no audit step); docs/stores.md:185-197 (checklist has no SCA box) | Policy plus an SCA step in release.yml's build job and a release checklist line (agent) | -| `osps_vm_05_03` | MUST [N/A J] | Unmet | Server dependencies are checked against known vulnerabilities on every pull request (mix deps.audit and mix hex.audit, inside a required check), but the npm lockfiles (root and site) are not scanned in CI, there is no check for malicious packages, and the blocking policy is not documented. Dependabot alerts are reported but do not block merges. | .github/workflows/ci.yml:72-73 (Hex only); no npm audit, OSV-Scanner or dependency-review job; gh api .../dependabot/alerts: 3 open npm alerts | Add an OSV-Scanner (covers MAL- malicious-package advisories) or dependency-review job over mix.lock, package-lock.json and site/package-lock.json with documented suppressions (agent); add it to the main ruleset's required checks (maintainer) | -| `osps_vm_06_01` | MUST [N/A J] | Unmet | Static analysis runs (Credo, ESLint with eslint-plugin-no-unsanitized, web-ext lint, ShellCheck) and CI fails on any finding, but there is no written policy defining a remediation threshold for SAST findings. | .github/workflows/ci.yml:70, 86, 147; eslint.config.js:44-47 | Write the SAST policy: which severities block, how a false positive is suppressed (with reason), deadlines (agent) | -| `osps_vm_06_02` | MUST [N/A J] | Unmet | Every pull request runs ESLint with eslint-plugin-no-unsanitized and a ban on innerHTML-style sinks, and fails on findings, but there is no security analyser for the Elixir server and no CodeQL, and the blocking policy is not documented. | eslint.config.js:10, 44-47; gh api .../code-scanning/default-setup: state not-configured; no Sobelow in server/mix.exs | Add a CodeQL workflow (javascript-typescript, actions) and Sobelow or Semgrep for Elixir with the policy from vm_06_01 (agent); make them required checks or enable a code-scanning merge rule in the main ruleset (maintainer) | +| `osps_vm_05_01` | MUST [N/A J] | Met | The dependency and static analysis policy sets the thresholds for dependency findings: what blocks a merge or a release by severity, deadlines to fix (critical 7 days and high 30, for runtime and dev-only alike; medium 60 days for runtime and 90 for dev-only), how an unreachable finding is accepted (osv-scanner.toml with a reason and an expiry date), and which dependency licenses are allowed (following the Apache Software Foundation's categories). https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/dependency-and-static-analysis-policy.md | `docs/security/dependency-and-static-analysis-policy.md`; `osv-scanner.toml` | – | +| `osps_vm_05_02` | MUST [N/A J] | Met | The release workflow runs the dependency scan before it builds anything: the build job needs the dependency-scan job (OSV-Scanner over every lockfile) to succeed, so a release can't be built while a known vulnerability is open and not accepted in osv-scanner.toml. The policy states the rule: https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/dependency-and-static-analysis-policy.md | `.github/workflows/release.yml` jobs `dependency-scan` and `build` (`needs`, `result == 'success'`); `.github/workflows/dependency-scan.yml` | – | +| `osps_vm_05_03` | MUST [N/A J] | Met | OSV-Scanner checks all three lockfiles (package-lock.json, site/package-lock.json, server/mix.lock) for known vulnerabilities and malicious packages on every pull request, every push to main and weekly; it is a required check on main, so a change can't merge with an open finding. mix deps.audit and mix hex.audit also run, and Dependabot alerts are on. https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/dependency-and-static-analysis-policy.md | `.github/workflows/dependency-scan.yml`; ruleset 24531669 requires `osv-scanner`; `.github/workflows/ci.yml` (`mix deps.audit`, `mix hex.audit`) | – | +| `osps_vm_06_01` | MUST [N/A J] | Met | The dependency and static analysis policy sets the thresholds for static analysis: any Sobelow finding fails, and a CodeQL alert of high or critical security severity, or of severity error, blocks the merge through the main ruleset's code scanning rule; ESLint and Credo fail on any finding, how a false positive is suppressed (per function, with a written reason), and the deadlines to fix. https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/dependency-and-static-analysis-policy.md | `docs/security/dependency-and-static-analysis-policy.md` | – | +| `osps_vm_06_02` | MUST [N/A J] | Met | Security analysis runs on every pull request and blocks the merge on a finding: Sobelow for the server (inside the required server check) and CodeQL with the security-extended queries for JavaScript and the workflows, whose alerts of high security severity or errors block the merge through the main ruleset's code scanning rule (and the analyses themselves are required checks). https://github.com/ScriptKittyOS/kotiko/blob/main/docs/security/dependency-and-static-analysis-policy.md | `.github/workflows/codeql.yml`; `.github/workflows/ci.yml` server job; ruleset 24531669 required checks | – | diff --git a/docs/security/dependency-and-static-analysis-policy.md b/docs/security/dependency-and-static-analysis-policy.md index a8daea4..0602be1 100644 --- a/docs/security/dependency-and-static-analysis-policy.md +++ b/docs/security/dependency-and-static-analysis-policy.md @@ -33,8 +33,8 @@ What runs: **Blocks a merge**: any finding above. They run on every pull request, and a pull request merges only when the required checks of the `main` ruleset pass. A new advisory against a version already on `main` turns every pull request red until it is fixed or accepted, so -nobody can merge around it. Not yet: the `osv-scanner` check is not in the ruleset's -required checks (the server job, with both mix audits, is). +nobody can merge around it. `osv-scanner` and the server job (with both mix audits) are +required checks. **Blocks a release**: the release workflow ([`release.yml`](../../.github/workflows/release.yml)) runs the same OSV-Scanner job before it builds anything; when it fails, nothing is built, @@ -55,14 +55,26 @@ These are the same targets as a reported vulnerability ([SECURITY.md](../../SECU "Fixed" means an updated or patched version; if upstream has no fix, we pin, patch or replace the package, or remove the code path that uses it. -**Licenses** (by review, not CI: OSV-Scanner has no license data for Hex packages). A -runtime dependency must be under Apache-2.0, MIT, BSD-2-Clause, BSD-3-Clause, ISC, 0BSD, -Zlib, Unicode-3.0, CC0-1.0 or public domain, all compatible with Kotiko's Apache-2.0; on -2026-10-06 every one is Apache-2.0 or MIT. Anything else (GPL, LGPL, AGPL, a custom or -missing license) needs a maintainer's decision recorded in -[slices/DECISIONS.md](../../slices/DECISIONS.md) before it merges. Dev-only tools may use -any [OSI-approved](https://opensource.org/licenses) license, since we don't distribute -them. The reviewer checks the license when a pull request adds a dependency or a +**Licenses** (by review, not CI: OSV-Scanner has no license data for Hex packages). The +rule follows the Apache Software Foundation's +[third-party license policy](https://www.apache.org/legal/resolved.html), the usual +reference for what an Apache-2.0 project may ship: + +- **Allowed** in anything Kotiko ships (the extension, the server, the shared spec data, + the docs site), the ASF's Category A: Apache-2.0, MIT and MIT-0, BSD-2-Clause, + BSD-3-Clause, ISC, 0BSD, Zlib, BSL-1.0, Unlicense, Unicode-3.0 (data), and CC0-1.0 or + public domain. On 2026-10-06 every server runtime dependency is Apache-2.0 or MIT, the + SQLite that exqlite bundles is public domain, and the shipped data is Unicode-3.0 (CLDR) + and MIT (stopwords-iso); the extension ships no npm code. +- **Allowed only unmodified, and labelled in NOTICE**, the ASF's Category B, each one + recorded in [slices/DECISIONS.md](../../slices/DECISIONS.md) before it merges: MPL-2.0 and + EPL-2.0 libraries, OFL-1.1 fonts, and CC-BY-4.0 media or data. +- **Not allowed** in anything Kotiko ships, the ASF's Category X: GPL, LGPL and AGPL (any + version), SSPL, BUSL, the Commons Clause and other field-of-use or non-commercial terms, + and code with no license or a custom one. + +Dev-only tools (tests, linters, build tools that put none of their code in what ships) +may use any [OSI-approved](https://opensource.org/licenses) license. The reviewer checks the license when a pull request adds a dependency or a Dependabot update changes one; a violation found later is replaced or approved before the next release. Each release's server SBOM (`kotiko-server-.cdx.json`) lists the runtime dependencies and their licenses. @@ -80,8 +92,9 @@ What runs, all on every pull request and push to `main`: **Blocks a merge**: any Sobelow, ESLint, web-ext, Credo or ShellCheck finding (their jobs are required checks). A CodeQL alert of security severity high or critical, or of severity -error, fails the "Code scanning results" check. Not yet: the `main` ruleset doesn't require -code scanning results, so until it does, CodeQL alerts are caught by review. +error, blocks the merge: the `main` ruleset requires code scanning results from CodeQL at +those thresholds (since 2026-10-06), and requires `osv-scanner`, `analyze (actions)` and +`analyze (javascript-typescript)` as checks. **Deadlines** for findings on `main` (a new CodeQL query, a new Sobelow release): an exploitable one is a vulnerability and follows the [SECURITY.md](../../SECURITY.md) diff --git a/scripts/bestpractices-links.mjs b/scripts/bestpractices-links.mjs index 50a9bb8..06785d3 100644 --- a/scripts/bestpractices-links.mjs +++ b/scripts/bestpractices-links.mjs @@ -6,8 +6,9 @@ // and writes a local page with one button per form section. A maintainer opens a link, // reviews the highlighted answers and saves; nothing is sent until they press Save. // -// node scripts/bestpractices-links.mjs # writes the page (passing, silver, gold) +// node scripts/bestpractices-links.mjs # the page: only answers that differ from the live entry // node scripts/bestpractices-links.mjs --baseline # also the OpenSSF Baseline levels +// node scripts/bestpractices-links.mjs --all # every answer, for a new entry // node scripts/bestpractices-links.mjs --out x.html // node scripts/bestpractices-links.mjs --check # validates the answers (CI) // @@ -18,6 +19,9 @@ import { join, resolve } from "node:path"; import { fileURLToPath, pathToFileURL } from "node:url"; export const SITE = "https://www.bestpractices.dev"; +// Kotiko's entry. The live answers are read from here, never from a URL built out of the +// document; --check fails if the document names a different entry. +export const PROJECT = "15259"; // The form's sections, in the order a maintainer fills them. export const SECTIONS = [ ["Passing", "passing"], @@ -96,6 +100,7 @@ export function parse(text) { export function validate({ project, groups }) { const problems = []; if (!project) problems.push("No bestpractices.dev project number found in the document."); + else if (project !== PROJECT) problems.push(`The document names entry ${project}, the script reads entry ${PROJECT}.`); const seen = new Map(); for (const g of groups) { for (const r of g.rows) { @@ -115,6 +120,19 @@ export function validate({ project, groups }) { } } } + // A criterion repeated at a higher metal level is the same field on the entry: saving one + // form overwrites the other, so every copy must say the same thing. + const metal = new Map(); + for (const g of groups) { + if (g.section.startsWith("baseline")) continue; + for (const r of g.rows) { + const first = metal.get(r.criterion); + if (!first) metal.set(r.criterion, { ...r, section: g.section }); + else if (first.status !== r.status || first.answer !== r.answer) { + problems.push(`${g.section} / ${r.criterion}: differs from its ${first.section} copy (one field on the entry)`); + } + } + } for (const link of links({ project: project ?? "0", groups })) { if (link.url.length > MAX_URL) problems.push(`${link.title}: one answer alone is too long for a link`); } @@ -159,41 +177,84 @@ export function links({ project, groups }) { return out; } +// The entry's own JSON names a Baseline field by its OSPS id (OSPS-AC-01.01_status); the form +// and the proposal links use the field name (osps_ac_01_01_status). +export const entryKey = (criterion) => + criterion.replace(/^osps_([a-z]{2})_(\d{2})_(\d{2})$/, (_, c, a, b) => `OSPS-${c.toUpperCase()}-${a}.${b}`); + +const same = (a, b) => String(a ?? "").replace(/\s+/g, " ").trim() === String(b ?? "").replace(/\s+/g, " ").trim(); + +// The answers that differ from what is saved on the entry (status or justification), per +// form, each with the saved status as `was`. With no entry, every answer. +export function changes({ groups }, entry = null) { + const bySection = new Map(); + for (const g of groups) { + for (const r of g.rows) { + const key = entryKey(r.criterion); + const was = entry ? entry[`${key}_status`] ?? "?" : null; + if (entry && was === r.status && same(entry[`${key}_justification`], r.answer)) continue; + if (!bySection.has(g.section)) bySection.set(g.section, []); + bySection.get(g.section).push({ ...r, was }); + } + } + return bySection; +} + +// As few links as fit for one form, each replacing what is saved (overrides). +export function formLinks(project, section, rows) { + const parts = []; + let part = []; + for (const r of rows) { + if (part.length && urlFor(project, section, [...part, r], true).length > MAX_URL) { + parts.push(part); + part = []; + } + part.push(r); + } + if (part.length) parts.push(part); + return parts.map((partRows) => ({ rows: partRows, url: urlFor(project, section, partRows, true) })); +} + const esc = (s) => String(s).replace(/[&<>"']/g, (c) => ({ "&": "&", "<": "<", ">": ">", '"': """, "'": "'" })[c]); -// The metal series (passing, silver, gold) is the badge the README shows; the Baseline levels -// are on the page only when asked for. -export function page({ project, groups }, { baseline = false } = {}) { - const all = links({ project, groups }).filter((l) => baseline || !l.section.startsWith("baseline")); - const sections = SECTIONS.map(([name, slug]) => ({ name, slug, links: all.filter((l) => l.section === slug) })) - .filter((s) => s.links.length); - const count = (rows, s) => rows.filter((r) => r.status === s).length; - const tally = (rows) => - STATUSES.map((s) => `${count(rows, s)} ${s}`).join(""); - const body = sections - .map((s) => { - const rows = s.links.flatMap((l) => l.rows); - const items = s.links +// The page: one card per form with the answers to update and one button per link. The metal +// series (passing, silver, gold) is the badge the README shows; Baseline only when asked for. +export function page(parsed, { entry = null, baseline = false, checked = "" } = {}) { + const { project } = parsed; + const pending = changes(parsed, entry); + const forms = SECTIONS.filter(([, slug]) => baseline || !slug.startsWith("baseline")); + const total = forms.reduce((n, [, slug]) => n + (pending.get(slug)?.length ?? 0), 0); + const cards = forms + .map(([name, slug]) => { + const rows = pending.get(slug) ?? []; + if (!rows.length) { + return `

${esc(name)}

Up to date: nothing to save.

`; + } + const parts = formLinks(project, slug, rows); + const buttons = parts .map((l, i) => { - const id = `${s.slug}-${i}`; - const list = l.rows - .map((r) => `
  • ${esc(r.criterion)} ${esc(r.status)} ${esc(r.answer)}
    • `) + const id = `${slug}-${i}`; + const label = parts.length > 1 ? `Open ${name} form, part ${i + 1} of ${parts.length}` : `Open ${name} form`; + const items = l.rows + .map((r) => { + const from = r.was === null ? "" : r.was === r.status ? "new wording" : `${r.was === "?" ? "blank" : r.was} → ${r.status}`; + return `
  • ${esc(r.criterion)} ${esc(r.status)} ${esc(from)}
    • `; + }) .join(""); - return `
  • - - ${tally(l.rows)} - Open and review -
    ${l.rows.length} answers
      ${list}
    -
    • `; + return `
    + ${esc(label)} + + +
    `; }) .join("\n"); - return `
    -

    ${esc(s.name)} ${tally(rows)}

    - -
    `; + return `

    ${esc(name)} ${rows.length} to update

    ${buttons}
    `; }) .join("\n"); + const intro = entry + ? `

    ${total ? `${total} answers on your entry differ from docs/best-practices.md.` : "Every answer on your entry already matches docs/best-practices.md."} Compared with the live entry${checked ? ` on ${esc(checked)}` : ""}; run the script again after saving to see what's left.

    ` + : `

    All answers in docs/best-practices.md, for a new entry.

    `; return ` @@ -201,59 +262,51 @@ export function page({ project, groups }, { baseline = false } = {}) { Kotiko badge answers

    Kotiko badge answers

    -

    Project ${esc(project)} on bestpractices.dev, prefilled from -docs/best-practices.md.

    +${intro}
      -
    1. Sign in to bestpractices.dev first (the links work after sign-in too, but this saves a redirect).
      2. -
    2. Press Open and review. The form opens with the proposed answers highlighted in yellow (🤖).
      3. -
    3. Read them, change anything you disagree with, then press Save at the bottom of the form.
      4. -
    4. Tick the box here so you know it's done. Nothing reaches bestpractices.dev until you save.
      5. +
    5. Sign in to bestpractices.dev.
      6. +
    6. Press a button. The form opens with our answers already put in, replacing what was saved.
      7. +
    7. Scroll to the bottom of the form and press Save. Then tick Saved here.
    - -${body} +${cards}
    @@ -273,10 +326,19 @@ if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.ur if (args.includes("--check")) { console.log(`docs/best-practices.md: ${rows} answers in ${parsed.groups.length} groups, all valid.`); } else { + let entry = null; + if (!args.includes("--all")) { + const res = await fetch(`${SITE}/projects/${PROJECT}.json`, { cache: "no-store" }); + if (!res.ok) { + console.error(`Couldn't read the entry (${res.status}). Use --all to list every answer instead.`); + process.exit(1); + } + entry = await res.json(); + } const i = args.indexOf("--out"); const out = resolve(i >= 0 ? args[i + 1] : join(tmpdir(), "kotiko-bestpractices.html")); - const html = page(parsed, { baseline: args.includes("--baseline") }); + const html = page(parsed, { entry, baseline: args.includes("--baseline"), checked: new Date().toISOString().slice(0, 16).replace("T", " ") + " UTC" }); writeFileSync(out, html); - console.log(`${html.match(/class="cta"/g).length} links. Open:\n${pathToFileURL(out).href}`); + console.log(`${(html.match(/class="cta"/g) ?? []).length} buttons. Open:\n${pathToFileURL(out).href}`); } } diff --git a/slices/DECISIONS.md b/slices/DECISIONS.md index 1543e74..4b2a58e 100644 --- a/slices/DECISIONS.md +++ b/slices/DECISIONS.md @@ -6,6 +6,15 @@ and why. Open questions live in each slice's own "Open questions" section and in ## 2026-10-06 +**Dependency licenses follow the Apache Software Foundation's categories.** *Lead agent, +delegated.* The maintainer: "you do this you are the coding agent i am the inventor" (on +confirming the license list in the dependency policy). Category A licenses (Apache-2.0, +MIT, BSD, ISC, 0BSD, Zlib, Unicode-3.0, CC0 and others) are allowed in anything Kotiko +ships; Category B (MPL-2.0, EPL-2.0, OFL-1.1 fonts, CC-BY media) only unmodified, labelled +and recorded here; Category X (GPL, LGPL, AGPL, SSPL, BUSL, non-commercial terms) never. +Source: . Everything Kotiko ships today is +Apache-2.0, MIT, public domain (SQLite) or Unicode-3.0 (CLDR data). + **Maintainers are repository collaborators, and governance changes like any other decision.** *Maintainer.* "That is not how I have it set up in my other projects so that is not how it should be done here. I cant add him to the team as an admin and there are no diff --git a/test/unit/bestpractices-links.test.mjs b/test/unit/bestpractices-links.test.mjs index d5c6639..6a7712e 100644 --- a/test/unit/bestpractices-links.test.mjs +++ b/test/unit/bestpractices-links.test.mjs @@ -5,11 +5,11 @@ import { test } from "node:test"; import assert from "node:assert/strict"; import { readFileSync } from "node:fs"; -import { MAX_URL, cells, links, page, parse, validate } from "../../scripts/bestpractices-links.mjs"; +import { MAX_URL, cells, changes, links, page, parse, validate } from "../../scripts/bestpractices-links.mjs"; const doc = (rows, heading = "## Passing: Basics") => `# Answers -- **Entry:** https://www.bestpractices.dev/en/projects/42 +- **Entry:** https://www.bestpractices.dev/en/projects/15259 ${heading} @@ -30,7 +30,7 @@ test("a row splits on pipes but keeps escaped ones", () => { test("reads the project number and only the form tables", () => { const parsed = parse(doc(["| `floss_license` | MUST | Met | Apache-2.0. | `LICENSE` | – |"])); - assert.equal(parsed.project, "42"); + assert.equal(parsed.project, "15259"); assert.deepEqual(parsed.groups, [ { section: "passing", name: "Basics", rows: [{ criterion: "floss_license", level: "MUST", status: "Met", answer: "Apache-2.0." }] }, ]); @@ -42,7 +42,7 @@ test("field names keep their case, as the form matches them exactly", () => { assert.deepEqual(validate(parsed), []); const [link] = links(parsed); assert.equal(new URL(link.url).searchParams.get("require_2FA_status"), "Met"); - assert.match(link.url, /\/projects\/42\/gold\/edit\?/); + assert.match(link.url, /\/projects\/15259\/gold\/edit\?/); }); test("the page shows the metal series unless Baseline is asked for", () => { @@ -55,6 +55,33 @@ test("the page shows the metal series unless Baseline is asked for", () => { assert.equal(page(parsed, { baseline: true }).match(/class="cta"/g).length, 2); }); +test("a criterion shared by two metal levels must have one answer", () => { + const text = doc(["| `crypto_weaknesses` | SHOULD | Met | SHA-256 only. | – | – |"]) + + "\n## Gold: Security\n\n| Criterion | Level | Status | Answer | Evidence | Needs |\n|---|---|---|---|---|---|\n" + + "| `crypto_weaknesses` | MUST | Met | No SHA-1 anywhere. | – | – |\n"; + assert.ok(validate(parse(text)).some((p) => /crypto_weaknesses: differs from its passing copy/.test(p))); +}); + +test("only answers that differ from the saved entry are listed, keyed the entry's way", () => { + const text = doc(["| `floss_license` | MUST | Met | Apache-2.0. | – | – |", "| `english` | SHOULD | Met | English. | – | – |"]) + + "\n## Baseline 1: General\n\n| Criterion | Level | Status | Answer | Evidence | Needs |\n|---|---|---|---|---|---|\n" + + "| `osps_ac_01_01` | MUST | Met | Org requires 2FA. | – | – |\n"; + const entry = { + floss_license_status: "Met", floss_license_justification: " Apache-2.0.\n", + english_status: "?", + "OSPS-AC-01.01_status": "Unmet", "OSPS-AC-01.01_justification": "old", + }; + const pending = changes(parse(text), entry); + assert.deepEqual(pending.get("passing").map((r) => [r.criterion, r.was]), [["english", "?"]]); + assert.deepEqual(pending.get("baseline-1").map((r) => [r.criterion, r.was]), [["osps_ac_01_01", "Unmet"]]); + assert.match(page(parse(text), { entry }), /Up to date|1 to update/); +}); + +test("the document must name the entry the script reads", () => { + const other = doc(["| `english` | SHOULD | Met | English. | – | – |"]).replace("projects/15259", "projects/99"); + assert.deepEqual(validate(parse(other)), ["The document names entry 99, the script reads entry 15259."]); +}); + test("baseline headings map to the baseline sections", () => { const parsed = parse(doc(["| `osps_ac_01_01` | MUST [N/A J] | Met | Org requires 2FA. | gh api | – |"], "## Baseline 1: Access Control")); assert.equal(parsed.groups[0].section, "baseline-1"); @@ -67,7 +94,7 @@ test("links carry each status and justification, and the force link adds overrid ])); const [link] = links(parsed); const url = new URL(link.url); - assert.equal(url.origin + url.pathname, "https://www.bestpractices.dev/en/projects/42/passing/edit"); + assert.equal(url.origin + url.pathname, "https://www.bestpractices.dev/en/projects/15259/passing/edit"); assert.equal(url.searchParams.get("floss_license_status"), "Met"); assert.equal(url.searchParams.get("floss_license_justification"), "Apache-2.0 & more."); assert.equal(url.searchParams.get("dco_status"), "Unmet"); @@ -99,11 +126,12 @@ test("validation names every problem", () => { assert.ok(problems.some((p) => /maintained: no answer/.test(p))); }); -test("the page escapes answers and links every group", () => { - const parsed = parse(doc(["| `english` | SHOULD | Met | Reports in English welcome. | – | – |"])); +test("the page escapes its links and has one button per link", () => { + const parsed = parse(doc(["| `english` | SHOULD | Met | Reports in English & more. | – | – |"])); const html = page(parsed); - assert.match(html, /Reports in <b>English<\/b> welcome\./); assert.equal(html.match(/class="cta"/g).length, 1); + assert.match(html, /href="https:\/\/www\.bestpractices\.dev\/en\/projects\/15259\/passing\/edit\?english_status=Met&/); + assert.doesNotMatch(html, /English<\/b>/); assert.match(html, /Kotiko badge answers<\/title>/); });