Skip to content

fix(db): resequence hygiene migration to 0028 (unblocks #195) - #237

Merged
Bindy-lbb merged 7 commits into
bytefolk:codex/fix-pr-185from
sun-970:fix/195-resequence-hygiene-0028
Oct 15, 2026
Merged

Bindy-lbb merged 7 commits into
bytefolk:codex/fix-pr-185from
sun-970:fix/195-resequence-hygiene-0028

Conversation

@sun-970

@sun-970 sun-970 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Why

@Bindy-lbb's review on #195 found a merge blocker: main commit 8c4080a (#235) added
0027_vault_sync.sql, colliding with the branch's 0027_data_plane_hygiene.sql.
Goose v3.22.1 CollectMigrations rejects the combined set with
goose: duplicate version 27 detected, which blocks database migration.

Because I only hold read access on bytefolk/mem, I cannot push to codex/fix-pr-185.
This PR carries the fix, based on the #195 head branch, so it can be merged or
cherry-picked in without rewriting published history.

Why this PR includes a main-integration merge

The diff is large (32 files) but only 6 files are the fix; the rest is the
merge of current main into this branch, which the fix depends on.

That dependency is not cosmetic. TestMigrationFilesContiguous requires
migrations to be numbered without gaps. On a branch that carries only the
rename, 0027 disappears:

migration_sequence_test.go:42: migration sequence gap:
  want 0027, got "0028_data_plane_hygiene.sql"

Integrating main (which owns 0027_vault_sync.sql) is what makes the sequence
26 -> 27 -> 28 contiguous again. Merging this PR therefore brings main in as
well, which is exactly what the review asked for.

The fix

Migration renumbering plus the runner constant:

File Change
server/internal/db/migrations/0027_data_plane_hygiene.sql renamed to 0028_data_plane_hygiene.sql
server/internal/db/migrations_test.go fixture path follows the rename
server/internal/db/migration_sequence_test.go version gates >= 27 -> >= 28
docs/MIGRATION_SEQUENCE.md predecessor schema 27, head 24 through 28, records the 0026 -> 0027 -> 0028 move
scripts/verify.sh EXPECTED_MIGRATION_HEAD=27 -> 28
docs/TESTING.md stale "declared current head (currently 27)" -> 28

0027 now belongs to main's vault_sync. Already released migration identities
are preserved; the unmerged draft is the only thing that moved. The hygiene DDL
itself is unchanged.

On scripts/verify.sh

The runner reads one constant. All nine assert_migration_version calls and the
up-to target already reference the variable, so the edit is a single line.
docs/TESTING.md carries the same stale number in prose and is updated with it
so the documented expectation matches the runner.

Verification (Go 1.27.0, Windows)

Reproduced with the repository-pinned Goose v3.22.1, same method as the review:

  • before - assembled tree (main 0027_vault_sync + branch 0027_data_plane_hygiene):
    panic: goose: duplicate version 27 detected
  • after - collected 28 migrations with no duplicate version, head 0028
  • the runner constant and the real head now agree at 28, which is the exact
    mismatch PostgreSQL reported as expected migration version 27, got 28
  • no remaining hardcoded 27 assertion: grep -rn 'EXPECTED_MIGRATION_HEAD=27'
    and the TESTING.md phrase both come back empty
  • bash -n scripts/verify.sh clean
go build ./...                                            # clean
go test ./internal/db/...                                 # pass
go test ./internal/memory/... ./internal/indexgeneration/... # pass
go test ./internal/api/...                                # pass

The one excluded test is TestWorkspaceImportSpoolsSecureTemporaryFile, which
asserts POSIX 0600 on a temp file. Windows cannot express that mode; it fails
identically on clean main and is unrelated to this PR.

Not claimed

No live PostgreSQL or production-latency run. Repro of the assertion mismatch is
local; the PostgreSQL integration job (including the race stage) still needs a
rerun on this head before re-review. TestMigrationUpgradeSequence needs a fresh
owned database and was not executed here.

Refs #195

sun-970 and others added 6 commits September 29, 2026 08:33
…tefolk#175) (bytefolk#224)

## Tracking record

Refs bytefolk#175. `live_producer.py` already landed via bytefolk#196; this PR closes
remaining measurable gaps that do not require a populated embedding
deployment.

## Summary

- `python3 -m benchmarks.recall produce` reads
engine/profile/provider/dimension from `GET /v1/version` and `GET
/v1/workspaces/current/ai-profile` (not CLI labels)
- default dataset is multilingual v1 (EN+ZH, including Chinese
image-description queries)
- `latency_ms` remains client `perf_counter` timing
- producer stays **on-demand**; CI/`make test-recall` stay hermetic
- structured-memory / unsupported filters are recorded but do not fail
the produce command

## Still open on bytefolk#175

No real-memd baseline is checked into `benchmarks/recall/baselines/`.
That needs a populated isolated memd + embedding provider run; inventing
one would be false evidence.

## Validation ledger

| ID | Criterion | Command | Result |
|---|---|---|---|
| V1 | Producer unit + discovery HTTP fixture | `python3 -m unittest
discover -s benchmarks/recall/tests` | PASS (48 tests) |

---------

Co-authored-by: Codex <codex@users.noreply.github.com>
Co-authored-by: 勒布朗-詹姆斯 <2986253039@qq.com>
…tefolk#235)

## Summary

Add workspace-scoped Vault discovery, snapshots and atomic delta
commits. Logical note IDs keep identical and empty notes distinct;
whole-Vault CAS rejects competing stale edits with 409. Immutable
revisions, mutable heads and tombstones preserve note identity and
history.

Closes bytefolk#234. Client integration: bytefolk/roleweave#575.

## Behavior and boundaries

- Existing workspace membership, read/write scopes and literal token
paths authorize every operation; properties remain untrusted data.
- Additive migration 0027 preserves existing file/auth/memory tables.
Downgrade drops Vault state and requires a backup.
- The snapshot is a separate current-state portability contract.
Existing workspace bundles and CLI/MCP/Web adapters do not transport
Vault history.
- No local login helpers, credentials, machine paths or production
deployment changes are included.

## CI baseline repair

The first run failed on unchanged vulnerable Web dependencies and an
unavailable public MinIO image, before exercising Vault code. Remove the
unpatched braces chain with official Tailwind/PostCSS 4 and
typescript-eslint 8; retain the existing theme and utility behavior with
browser compatibility tests. Build test-only MinIO/mc images from
checksum-pinned official sources and preserve isolation and every
lifecycle assertion. Audit thresholds, checks and timeouts remain
unchanged.

## Validation

Local PASS: complete Go build/vet/unit/race; full disposable PostgreSQL
integration and integration-race; clean Web install, zero-finding
Linux/Windows audits, 109 unit tests, lint/typecheck/build, all browser
acceptance; complete HTTP/CLI/MCP lifecycle with CI-exact Go 1.25.0.
Test resources were removed.

Remote verification on `18db8a26791b3827b86e8d3c9e11fec5ac171999`: all
21 checks and all 12 required checks pass, including [CI and Windows
audit](https://github.com/bytefolk/mem/actions/runs/37809406457) and
[full PostgreSQL/HTTP/CLI/MCP/browser
validation](https://github.com/bytefolk/mem/actions/runs/37809406428).
Non-author CODEOWNERS approval remains required. Public advisory
sources, compatibility checks and exact evidence: [validation
ledger](https://github.com/bytefolk/mem/blob/18db8a26791b3827b86e8d3c9e11fec5ac171999/docs/acceptance/VAULT_SYNC_VALIDATION.md).
Bumps [pypdf](https://github.com/py-pdf/pypdf) from 6.18.0 to 6.19.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/py-pdf/pypdf/releases">pypdf's
releases</a>.</em></p>
<blockquote>
<h2>Version 6.19.0, 2026-09-16</h2>
<h2>What's new</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Limit size of alphabetical page labels (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4096">#4096</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>Deprecations (DEP)</h3>
<ul>
<li>Replace PdfWriter method add_js (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3979">#3979</a>)
by <a href="https://github.com/j-t-1"><code>@​j-t-1</code></a></li>
</ul>
<h3>Performance Improvements (PI)</h3>
<ul>
<li>Move static value out of loop body for appearance stream data (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4087">#4087</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
<li>Reduce number of full data lookups for attachment mapping API (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4081">#4081</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>Bug Fixes (BUG)</h3>
<ul>
<li>Do not copy unrelated pages when appending pages with non-terminal
fields (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4078">#4078</a>)
by <a
href="https://github.com/anandghegde"><code>@​anandghegde</code></a></li>
<li>Use page reference for existing internal link targets (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4076">#4076</a>)
by <a href="https://github.com/r-kamei"><code>@​r-kamei</code></a></li>
<li>Arabic-Indic digits are reversed during text extraction (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4077">#4077</a>)
by <a
href="https://github.com/Syamjith-NK"><code>@​Syamjith-NK</code></a></li>
<li>Parse a string rect for add_uri into a rectangle (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4074">#4074</a>)
by <a
href="https://github.com/RavSinghChandan"><code>@​RavSinghChandan</code></a></li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.18.1...6.19.0">Full
Changelog</a></p>
<h2>Version 6.18.1, 2026-09-11</h2>
<h2>What's new</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Further restrict FlateDecode recovery (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4073">#4073</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
<li>Limit entry count for TrueType and Type1 font <code>/Widths</code>
(<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4072">#4072</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
<li>Limit allowed length of tokens in parse_bfchar (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4071">#4071</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>Bug Fixes (BUG)</h3>
<ul>
<li>Use current text matrix for visitor_text (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4062">#4062</a>)
by <a href="https://github.com/r-kamei"><code>@​r-kamei</code></a></li>
<li>Repeat the letter for /S /A and /S /a page labels past Z (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4065">#4065</a>)
by <a
href="https://github.com/youdie006"><code>@​youdie006</code></a></li>
<li>Use font color for FreeText default appearance (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4051">#4051</a>)
by <a
href="https://github.com/Yuki9814"><code>@​Yuki9814</code></a></li>
</ul>
<h3>Robustness (ROB)</h3>
<ul>
<li>Fix compatibility with fonttools &lt; 4.58.0 (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4050">#4050</a>,
<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4059">#4059</a>)
by <a href="https://github.com/MeggyCal"><code>@​MeggyCal</code></a> and
<a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>Documentation (DOC)</h3>
<ul>
<li>Use combined matrix in visitor examples (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4066">#4066</a>)
by <a href="https://github.com/r-kamei"><code>@​r-kamei</code></a></li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.18.0...6.18.1">Full
Changelog</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/py-pdf/pypdf/blob/main/CHANGELOG.md">pypdf's
changelog</a>.</em></p>
<blockquote>
<h2>Version 6.19.0, 2026-09-16</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Limit size of alphabetical page labels (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4096">#4096</a>)</li>
</ul>
<h3>Deprecations (DEP)</h3>
<ul>
<li>Replace PdfWriter method add_js (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3979">#3979</a>)</li>
</ul>
<h3>Performance Improvements (PI)</h3>
<ul>
<li>Move static value out of loop body for appearance stream data (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4087">#4087</a>)</li>
<li>Reduce number of full data lookups for attachment mapping API (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4081">#4081</a>)</li>
</ul>
<h3>Bug Fixes (BUG)</h3>
<ul>
<li>Do not copy unrelated pages when appending pages with non-terminal
fields (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4078">#4078</a>)</li>
<li>Use page reference for existing internal link targets (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4076">#4076</a>)</li>
<li>Arabic-Indic digits are reversed during text extraction (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4077">#4077</a>)</li>
<li>Parse a string rect for add_uri into a rectangle (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4074">#4074</a>)</li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.18.1...6.19.0">Full
Changelog</a></p>
<h2>Version 6.18.1, 2026-09-11</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Further restrict FlateDecode recovery (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4073">#4073</a>)</li>
<li>Limit entry count for TrueType and Type1 font <code>/Widths</code>
(<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4072">#4072</a>)</li>
<li>Limit allowed length of tokens in parse_bfchar (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4071">#4071</a>)</li>
</ul>
<h3>Bug Fixes (BUG)</h3>
<ul>
<li>Use current text matrix for visitor_text (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4062">#4062</a>)</li>
<li>Repeat the letter for /S /A and /S /a page labels past Z (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4065">#4065</a>)</li>
<li>Use font color for FreeText default appearance (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4051">#4051</a>)</li>
</ul>
<h3>Robustness (ROB)</h3>
<ul>
<li>Fix compatibility with fonttools &lt; 4.58.0 (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4050">#4050</a>,
<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4059">#4059</a>)</li>
</ul>
<h3>Documentation (DOC)</h3>
<ul>
<li>Use combined matrix in visitor examples (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4066">#4066</a>)</li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.18.0...6.18.1">Full
Changelog</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/py-pdf/pypdf/commit/d62cb58d3988b291b0435eddfd118c4f8f6b6a46"><code>d62cb58</code></a>
REL: 6.19.0</li>
<li><a
href="https://github.com/py-pdf/pypdf/commit/0d8b5a8832cde1ba308b5c27567b879b7b7eed4a"><code>0d8b5a8</code></a>
SEC: Limit size of alphabetical page labels (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4096">#4096</a>)</li>
<li><a
href="https://github.com/py-pdf/pypdf/commit/959467a9b95be83e2dc5ae873ece5f371263ede7"><code>959467a</code></a>
PI: Move static value out of loop body for appearance stream data (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4087">#4087</a>)</li>
<li><a
href="https://github.com/py-pdf/pypdf/commit/20822b263f063c96ad535405f5b8c4d81768950b"><code>20822b2</code></a>
DEV: Bump zizmorcore/zizmor-action from 0.6.3 to 0.6.4 (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4093">#4093</a>)</li>
<li><a
href="https://github.com/py-pdf/pypdf/commit/25f230191b89cdcf2b4ba3d8051d52c7392831d0"><code>25f2301</code></a>
DEP: Replace PdfWriter method add_js (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3979">#3979</a>)</li>
<li><a
href="https://github.com/py-pdf/pypdf/commit/a92443848fda10cc2a65a7b6397fd8e1986c17d2"><code>a924438</code></a>
BUG: Do not copy unrelated pages when appending pages with non-terminal
field...</li>
<li><a
href="https://github.com/py-pdf/pypdf/commit/cf5cec2e49b9da8fae0d82a0f99f1bfe9eceb8ed"><code>cf5cec2</code></a>
BUG: Use page reference for existing internal link targets (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4076">#4076</a>)</li>
<li><a
href="https://github.com/py-pdf/pypdf/commit/96d81f05ba4b3ec9ff1b3c3e52bc71a3cdd702c7"><code>96d81f0</code></a>
BUG: Arabic-Indic digits are reversed during text extraction (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4077">#4077</a>)</li>
<li><a
href="https://github.com/py-pdf/pypdf/commit/2d219015b4cf92edcfd6d28c564295f0704fd204"><code>2d21901</code></a>
DEV: Fix Color class for latest mypy (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4082">#4082</a>)</li>
<li><a
href="https://github.com/py-pdf/pypdf/commit/893a01002aa8e23b3a61ae4b38ab74edbc904ab9"><code>893a010</code></a>
MAINT: Split PdfDocCommon._flatten (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/4070">#4070</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/py-pdf/pypdf/compare/6.18.0...6.19.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pypdf&package-manager=uv&previous-version=6.18.0&new-version=6.19.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts page](https://github.com/bytefolk/mem/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: 修雨 <47820304+PeterGuy326@users.noreply.github.com>
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.7.0 to 2.8.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/urllib3/urllib3/releases">urllib3's
releases</a>.</em></p>
<blockquote>
<h2>2.8.0</h2>
<h2>🚀 urllib3 is fundraising for HTTP/2 support</h2>
<p><a
href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3
is raising ~$40,000 USD</a> to release HTTP/2 support and ensure
long-term sustainable maintenance of the project. If your company or
organization uses Python and would benefit from HTTP/2 support in
Requests, pip, cloud SDKs, and thousands of other projects <a
href="https://opencollective.com/urllib3">please consider contributing
financially</a> to ensure HTTP/2 support is developed sustainably and
maintained for the long-haul.</p>
<p>Thank you for your support.</p>
<h2>Security</h2>
<p>Fixed the following security issues:</p>
<ul>
<li>The TLS configuration for HTTPS proxies could be ignored or
overridden. (High severity, GHSA-8988-9cw3-xx77)</li>
<li><code>HTTPResponse.stream()</code> and <code>read_chunked()</code>
could buffer a chunk-size line of unbounded length in memory. (High
severity, GHSA-vxq7-64xx-v4gw)</li>
<li>Chunked Deflate streaming could enter an infinite loop. (Medium
severity, GHSA-gh4c-6fx4-qh6g)</li>
</ul>
<blockquote>
<p>[!IMPORTANT]
urllib3 2.8.0 fixes HTTPS proxy TLS configuration being ignored or
overridden by destination settings. Configurations relying on that
behavior may require changes.</p>
<p>Configure proxy CA certificates and client certificates in
<code>proxy_ssl_context</code>, and proxy identity checks with
<code>proxy_assert_hostname</code> or
<code>proxy_assert_fingerprint</code>. Destination client certificates
and identity overrides no longer apply to HTTPS forwarding proxy
connections.</p>
</blockquote>
<blockquote>
<p>[!NOTE]
CVE IDs had not yet been assigned to these advisories at the time of
release due to a backlog at GitHub's CNA.</p>
</blockquote>
<h2>Deprecations &amp; Removals</h2>
<ul>
<li>Deprecated using an empty collection as the <code>Retry</code>
option <code>allowed_methods</code> to retry any verb. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5044">#5044</a>)</li>
</ul>
<h2>Features</h2>
<ul>
<li>Added <code>Url.auth_decoded</code> and
<code>Url.auth_decoded_joined</code> convenience properties to the
result of <code>parse_url()</code>. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4945">#4945</a>)</li>
<li>Added <code>basic_auth_encoding</code> and
<code>proxy_basic_auth_encoding</code> parameters to
<code>urllib3.util.make_headers()</code>. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5092">#5092</a>)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>
<p>Fixed response header handling to replace obsolete folded header
lines (<code>obs-fold</code>) with spaces in accordance with RFC 9112,
preventing raw CRLF sequences from appearing in header values such as
<code>Set-Cookie</code>. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/1362">#1362</a>)</p>
</li>
<li>
<p>Fixed usage of <code>proxy_ssl_context</code> with
<code>ProxyManager</code> when
<code>use_forwarding_for_https=True</code>. Passing
<code>ssl_context</code> instead of <code>proxy_ssl_context</code> for
HTTPS proxies in this configuration now emits a
<code>FutureWarning</code> and will raise an error in v3.0. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/2577">#2577</a>)</p>
</li>
<li>
<p>Changed behavior of the default <code>ConnectionPool.pool</code>
initialization. <code>LifoQueue</code> is now resolved from the
<code>queue</code> module after the <code>ConnectionPool</code> is
instantiated instead of using the default cached <code>QueueCls</code>
class property. This is done because sometimes the
<code>queue.LifoQueue</code> is monkey-patched late in the program, such
as by gevent. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3289">#3289</a>)</p>
</li>
<li>
<p>Raised <code>UnrewindableBodyError</code> instead of
<code>ValueError</code> when retrying a request whose body had
<code>tell()</code> but not <code>seek()</code>. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3779">#3779</a>)</p>
</li>
<li>
<p>Decoded percent-encoded SOCKS proxy credentials before authenticating
with the proxy server. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3785">#3785</a>)</p>
</li>
<li>
<p>Fixed <code>HTTPResponse.drain_conn()</code> to discard unread
response data in 64 KiB chunks (same as the default <code>amt</code>
when doing <code>HTTPResponse.stream(...)</code>). (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5019">#5019</a>)</p>
</li>
<li>
<p>Fixed <code>is_ipaddress()</code> to detect non-standard IPv4 forms
accepted by <code>socket.connect</code>, such as hex
(<code>0x7f000001</code>), octal (<code>0177.0.0.1</code>), and decimal
integers (<code>2130706433</code>), ensuring SSL certificate
verification uses the correct mode for these addresses. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5029">#5029</a>)</p>
</li>
<li>
<p>Fixed <code>HTTPConnectionPool.urlopen</code> raising a misleading
<code>FullPoolError</code> instead of <code>ValueError</code> when
called with an invalid <code>timeout</code> argument on a pool created
with <code>block=True</code>. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5059">#5059</a>)</p>
</li>
<li>
<p>Fixed port-zero handling to preserve explicit <code>:0</code> values
instead of substituting the default ports 80 or 443 in URL parsing, pool
selection, proxy configuration, <code>connection_from_url()</code>, and
HTTP/2 request authority. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5071">#5071</a>,
<a
href="https://redirect.github.com/urllib3/urllib3/issues/5101">#5101</a>)</p>
</li>
<li>
<p>Fixed a bug where <code>PoolManager</code> passed the
<code>assert_hostname</code> and <code>assert_fingerprint</code>
parameters to HTTP connection pools. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5077">#5077</a>)</p>
</li>
<li>
<p>Fixed <code>HTTPConnectionPool.urlopen()</code> and HTTP proxy
forwarding to strip URL fragments from absolute request targets before
sending requests. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5079">#5079</a>)</p>
</li>
<li>
<p>Added safeguards to the proxy tunneling code to prevent potential
security issues when handling invalid characters in the proxy host and
HTTP headers. This change affects users of Python 3.10, Python 3.11, and
Python 3.12 when the standard library does not contain the fix; those on
newer Python versions should upgrade to 3.13.14+ or 3.14.5+ to get the
same security fixes. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5091">#5091</a>)</p>
</li>
<li>
<p>Fixed <code>HTTPSConnection.connect()</code> overriding
<code>ProxyConfig.ssl_context</code>'s certificate policy and proxy
identity checks with the target connection's TLS settings when
forwarding through an HTTPS proxy.</p>
<p><code>HTTPSConnection</code> no longer applies target SNI,
assertions, or client credentials to forwarding proxy handshakes and
continues to use its <code>ssl_context</code> as a fallback when an
HTTPS proxy forwards an HTTP target. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5093">#5093</a>)</p>
</li>
<li>
<p>Fixed URL parsing to more strictly enforce RFC 3986 host syntax,
rejecting invalid host input such as raw spaces and control characters,
malformed percent-encodings, and percent-encoded control characters in
HTTP(S) hosts and IPv6 zone identifiers, including proxy CONNECT tunnel
targets. Host normalization now also follows RFC 3986 normalization
rules for percent-encoded octets by decoding percent-encoded unreserved
characters and uppercasing the hexadecimal digits of retained
percent-encoded octets. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5095">#5095</a>)</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/urllib3/urllib3/blob/main/CHANGES.rst">urllib3's
changelog</a>.</em></p>
<blockquote>
<h1>2.8.0 (2026-09-15)</h1>
<h2>Security</h2>
<p>Fixed the following security issues:</p>
<ul>
<li>The TLS configuration for HTTPS proxies could be ignored or
overridden.
(High severity, <code>GHSA-8988-9cw3-xx77
&lt;https://github.com/urllib3/urllib3/security/advisories/GHSA-8988-9cw3-xx77&gt;</code>__)</li>
<li><code>HTTPResponse.stream()</code> and <code>read_chunked()</code>
could buffer a chunk-size
line of unbounded length in memory. (High severity,
<code>GHSA-vxq7-64xx-v4gw
&lt;https://github.com/urllib3/urllib3/security/advisories/GHSA-vxq7-64xx-v4gw&gt;</code>__)</li>
<li>Chunked Deflate streaming could enter an infinite loop. (Medium
severity,
<code>GHSA-gh4c-6fx4-qh6g
&lt;https://github.com/urllib3/urllib3/security/advisories/GHSA-gh4c-6fx4-qh6g&gt;</code>__)</li>
</ul>
<p>.. caution::</p>
<pre><code>urllib3 2.8.0 fixes HTTPS proxy TLS configuration being
ignored or
overridden by destination settings. Configurations relying on that
behavior may require changes.
<p>Configure proxy CA certificates and client certificates in
<code>proxy_ssl_context</code>, and proxy identity checks with
<code>proxy_assert_hostname</code> or
<code>proxy_assert_fingerprint</code>.
Destination client certificates and identity overrides no longer
apply to HTTPS forwarding proxy connections.
</code></pre></p>
<h2>Deprecations &amp; Removals</h2>
<ul>
<li>Deprecated using an empty collection as the <code>Retry</code>
option
<code>allowed_methods</code> to retry any verb.
(<code>[#5044](urllib3/urllib3#5044)
&lt;https://github.com/urllib3/urllib3/issues/5044&gt;</code>__)</li>
</ul>
<h2>Features</h2>
<ul>
<li>Added <code>Url.auth_decoded</code> and
<code>Url.auth_decoded_joined</code> convenience
properties to the result of <code>parse_url()</code>.
(<code>[#4945](urllib3/urllib3#4945)
&lt;https://github.com/urllib3/urllib3/issues/4945&gt;</code>__)</li>
<li>Added <code>basic_auth_encoding</code> and
<code>proxy_basic_auth_encoding</code> parameters to
<code>urllib3.util.make_headers()</code>.
(<code>[#5092](urllib3/urllib3#5092)
&lt;https://github.com/urllib3/urllib3/issues/5092&gt;</code>__)</li>
</ul>
<h2>Bugfixes</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/urllib3/urllib3/commit/b1d30ab61fe0db8f11092805e8c5ac43e091064a"><code>b1d30ab</code></a>
Release 2.8.0</li>
<li><a
href="https://github.com/urllib3/urllib3/commit/9016d7e8afc68185496ef07f3c3a4a743d04922e"><code>9016d7e</code></a>
Skip <code>test_read_chunked_with_trailing_data_does_not_hang</code> for
brotlicffi (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5258">#5258</a>)</li>
<li><a
href="https://github.com/urllib3/urllib3/commit/9101f581a8b3659af23b6ff335ae77200ca33533"><code>9101f58</code></a>
Fix <code>nox -s docs</code> warning (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5256">#5256</a>)</li>
<li><a
href="https://github.com/urllib3/urllib3/commit/cd770b059b543be29298ea5c52afb0b1b090f5ed"><code>cd770b0</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/urllib3/urllib3/commit/ea2ad7b21a80da3632f80016526a18864586077f"><code>ea2ad7b</code></a>
Merge commit from fork</li>
<li><a
href="https://github.com/urllib3/urllib3/commit/0716e31534345dc1599ea95d903c79f276239bd8"><code>0716e31</code></a>
Fix loading unencrypted client keys with a password in pyOpenSSL (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5255">#5255</a>)</li>
<li><a
href="https://github.com/urllib3/urllib3/commit/43c68c8b43a9dcb44ed2cf4ec91384ca0d46b37d"><code>43c68c8</code></a>
Test pickling of <code>InvalidChunkLength</code> (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5247">#5247</a>)</li>
<li><a
href="https://github.com/urllib3/urllib3/commit/308b279b3fb28e7bee952e152ec5baeb5bfd0817"><code>308b279</code></a>
Share security policy between GitHub and Read the Docs (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5253">#5253</a>)</li>
<li><a
href="https://github.com/urllib3/urllib3/commit/53fa0731b27d4b71ab0755ea5b896422d005d706"><code>53fa073</code></a>
Add policy on duplicate pull requests (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5252">#5252</a>)</li>
<li><a
href="https://github.com/urllib3/urllib3/commit/5f2a6a843d0100d1351c3f94d58581ca98d17267"><code>5f2a6a8</code></a>
Assert on the ALPN extension in test_tunnel_sets_http_11_alpn (<a
href="https://redirect.github.com/urllib3/urllib3/issues/5232">#5232</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/urllib3/urllib3/compare/2.7.0...2.8.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=uv&previous-version=2.7.0&new-version=2.8.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts page](https://github.com/bytefolk/mem/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: 修雨 <47820304+PeterGuy326@users.noreply.github.com>
main commit 8c4080a (bytefolk#235) added 0027_vault_sync.sql, colliding with this
branch's 0027_data_plane_hygiene.sql. Goose v3.22.1 CollectMigrations rejects
the combined set with "goose: duplicate version 27 detected", which blocks
database migration; the previous 21 green checks did not cover this new
combination.

Integrate current main and move the unmerged hygiene migration to the next
available version, 0028:

- rename migrations/0027_data_plane_hygiene.sql -> 0028_data_plane_hygiene.sql
- update migrations_test.go fixture path
- update TestMigrationUpgradeSequence version gates (>= 28) so the upgrade
  sequence still asserts 26 producer-agent indexing, 27 vault_sync, then 28
  dedupe/unique rejection
- update docs/MIGRATION_SEQUENCE.md: predecessor schema 27, head 24 through 28,
  and record why the version moved twice (0026 -> 0027 -> 0028)

Already released migration identities are preserved; 0027 now belongs to
main's vault_sync.

Verification on this head (Go 1.27.0, Windows):
- goose.CollectMigrations over the assembled tree: 28 migrations, no duplicate
  version, head 0028 (was: duplicate version 27 panic)
- go build ./... clean
- go test ./internal/db/... ./internal/memory/... ./internal/indexgeneration/...
  ./internal/api/... pass (excluding the pre-existing Windows-only
  TestWorkspaceImportSpoolsSecureTemporaryFile 0600-mode failure, which also
  fails on clean main)

@Bindy-lbb Bindy-lbb left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head e4e085fff920e3da180dfc5b603508de2b757696.

The correction about integrating the whole PR is valid: d7a16a7 includes current main (73c316a), supplying 0027_vault_sync.sql; cherry-picking only the rename onto #195 would leave a gap. I verified the 4 fix files and verified that the 28 inherited paths match main (scope: 32 files, 4 reviewed as the new fix, 28 inherited upstream paths checked for identity rather than independently re-reviewed).

Locally, the pinned Goose v3.22.1 reports collected 28 migrations; head=0028; TestMigrationFilesContiguous and go test ./internal/db ./internal/memory ./internal/indexgeneration ./internal/api pass. These local package checks are not live PostgreSQL acceptance.

One remaining blocker: scripts/verify.sh:7 still says EXPECTED_MIGRATION_HEAD=27. PostgreSQL CI on this exact head has already successfully upgraded the populated sequence through 28, then fails the runner assertion with ERROR: expected migration version 27, got 28 (run 38694417311, PostgreSQL integration job 116287630741). This also prevents the subsequent rollback/forward/regression and race stages from proving the updated migration path.

Please update this one runner constant alongside the existing four-file fix:

-EXPECTED_MIGRATION_HEAD=27
+EXPECTED_MIGRATION_HEAD=28

Then rerun PostgreSQL integration (including the race stage) on the updated head before merging the whole #237 into codex/fix-pr-185. Preserve the main-integration history as described; do not cherry-pick only the rename. I have also requested a fresh PostgreSQL job rerun on the current head; an unchanged-head rerun does not fix this deterministic version mismatch.

No new issue found in the migration rename or its sequence gates. Withholding approval/merge until the runner matches head 28 and PostgreSQL CI passes.

@sun-970
sun-970 force-pushed the fix/195-resequence-hygiene-0028 branch from 0a2a588 to 20fa7af Compare October 15, 2026 07:39
PostgreSQL integration on the previous head upgraded the populated sequence
through 28 successfully, then failed the runner assertion with
"ERROR: expected migration version 27, got 28" (run 38694417311, PostgreSQL
integration job 116287630741), which also kept the rollback/forward and race
stages from running.

The runner reads one constant, so this is the whole fix:

- scripts/verify.sh: EXPECTED_MIGRATION_HEAD 27 -> 28. All nine
  assert_migration_version calls and the up-to target already reference the
  variable, so no other line changes. Mode stays 100755.
- docs/TESTING.md: the same stale "declared current head (currently 27)" is
  updated to 28 so the documented expectation matches the runner.

Checked there is no remaining hardcoded 27 migration assertion:
grep -rn 'EXPECTED_MIGRATION_HEAD=27' and the TESTING.md phrase both come back
empty. bash -n scripts/verify.sh is clean.

Not claimed: no local PostgreSQL run. The deterministic mismatch this repairs
is the assertion, not the migration path; the PostgreSQL integration and race
stages still need a rerun on the updated head.
@sun-970
sun-970 force-pushed the fix/195-resequence-hygiene-0028 branch from 20fa7af to 8263a01 Compare October 15, 2026 07:41
@sun-970

sun-970 commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

PostgreSQL integration failed at 28s on 20fa7af with:

./scripts/verify.sh: Permission denied
Process completed with exit code 126.

That was my fault, not a migration problem. Pushing through the GitHub API, I
built the tree entry for scripts/verify.sh with a hardcoded mode: "100644",
which stripped the executable bit the script needs. Corrected on 8263a01:
the entry is back to 100755 (tree API confirms 100755 scripts/verify.sh,
alongside the already-executable scripts/acceptance_agent_memory.sh).

Two further notes for the review, both self-inflicted and now clean:

  • The first API push also carried my working tree's CRLF into scripts/verify.sh
    and docs/TESTING.md, which showed up as ~470-line churn in each. Re-pushed
    from the repo's own LF blob at e4e085f, so verify.sh is now a single
    +1/-1 hunk and TESTING.md is the intended prose change only. Total diff is
    back to 3059/1118.
  • Both pushes went through the API because git-over-https is being reset in my
    environment right now; that is a transport workaround only and does not affect
    the content or history shape of this branch.

The runner constant and the real head now agree at 28, which is the mismatch
PostgreSQL reported. CI on 8263a01 is green except this job, which is still in
progress. @Bindy-lbb a rerun on this head is still the gate - I have not run
PostgreSQL locally.

@Bindy-lbb Bindy-lbb left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fresh review on exact head 8263a012e905e1bcff32608538438cd18b5857d3: both prior blockers are resolved. No blocking finding.

  • Current main 73c316a is an ancestor via d7a16a7; the complete branch supplies vault migration 0027 and hygiene migration 0028, so no isolated rename/cherry-pick is appropriate.
  • The six fix files match the intended scope. Compared with my previously reviewed e4e085f, only two lines change: runner head 27 -> 28 and the testing documentation head 27 -> 28. The migration SQL is a byte-identical rename.
  • Verified scripts/verify.sh mode 100755, LF bytes, shell syntax, and EXPECTED_MIGRATION_HEAD=28 matching the collected migration head.
  • Local pinned Goose v3.22.1 collects 28 migrations without duplicate versions; db/memory/indexgeneration/api package tests pass.
  • All 21 hosted checks on this exact SHA succeed. Run 38700472109 explicitly records both Migration and PostgreSQL regression and PostgreSQL race regression as successful. The live PostgreSQL evidence is from CI, not a claimed local PostgreSQL run.

Review coverage: both files changed since the previous review inspected (2/2, 100%, no skips); all six fix files rechecked. The inherited current-main files retain the prior upstream-identity verification scope.

Approved for full-branch integration into codex/fix-pr-185, preserving main ancestry; no repository protection or merge-method settings should be weakened.

@Bindy-lbb
Bindy-lbb merged commit 8263a01 into bytefolk:codex/fix-pr-185 Oct 15, 2026
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants