Conversation
podup is consumed as a program: epistle and helmly install it as a required package and run the binary, and nothing outside this repository links the crate. Several comments and one user doc still described it as a library other products use: - `internal/lib.rs`: the crate docs called it "the docker-compose to Podman translator library", and the `deny(missing_docs)` rationale cited "a crate two other products consume as a library". - `internal/engine/mod.rs`: a design note justified itself with "podup is consumed as a library". - `internal/compose/mod.rs`, `diagnostics/mod.rs`, `mod_tests.rs`: comments about "library consumers", one naming an old product. - `docs/docker-migration.md`: told readers how to embed podup as a library. The crate docs now say the library target is internal code and that the supported interface is the command line (commands, flags, exit codes, `--format json`). Text only; no behaviour changes. The full non-live suite passed (2950). Signed-off-by: Jaro-c <75870284+Jaro-c@users.noreply.github.com> Signed-off-by: Jaro-c <75870284+Jaro-c@users.noreply.github.com>
…ds its body (#1927) `podup logs` took twice as long as reading the same endpoint with `curl`. Podman sends every log line as its own HTTP chunk, and the streaming client ran the hyper connection in a spawned task that handed each decoded frame to the reader through a channel, so every line cost a cross-task wake-up: about 6 M `futex` calls on the fixture below. The streaming open path now returns the connection future instead of spawning it. `send_streaming` polls it together with the response head, and the response body (`DrivenBody`) polls it before each frame, so the frames hyper has just decoded are read in the same task without a wake-up. Dropping the body drops the connection and closes the socket, as aborting the driver task did before. Buffered requests and the pool are unchanged. Two details keep the old behaviour: - A connection future that has already finished is never polled again (`ConnState`). A short response sent with `Connection: close` can finish the connection in the same poll that delivers the head. - A connection error is logged at debug level and not returned from the body. hyper queues its own error into the body behind the frames it already decoded, so the caller reads every delivered line and then the failure, in the order it did before. The spawned driver discarded the connection's result too (`let _ = conn.await;`). Measured on a container holding 2,097,152 empty lines plus `seq 1 20000` (23,377,566 bytes of output) with `podup -p flood logs --no-color --tail all`, three runs each, release builds of develop and of this branch from the same toolchain: | | wall | user | sys | |---|---|---|---| | develop | 20.3 to 20.7 s | 10.6 to 10.9 s | 22.0 to 22.3 s | | this branch | 12.3 s | 4.1 to 4.2 s | 8.6 to 8.7 s | | `curl` on the same endpoint | 10.5 s | 2.5 s | 4.5 s | Every run printed the same bytes (sha256 `01562b1396e18d1d0fc7b0c8bd35f26b6fd1bc14aeb95f93e0c9b4ebe9fd4cd7`); the published 5.10.1 binary gives the same hash in 21.0 s. How I checked it did not regress: - The full live suite on Podman 5.7.0, one run at a time: 238 of 238, twice (before and after the last fix). - develop's binary and this branch's binary ran the same 26 commands on one compose file: `up`, `run --rm` and `run -d`, `exec` with stdin, an exit code, `-e` and `-w`, `cp` of a file and of a directory, `stats --format json`, `logs` with timestamps, with `--tail` and with `-f`, `wait`, `restart`, scale up and down, `ps --format json`, `export`, `commit`, `pull`, `top`, `ls`, `kill -s`, `rm`, `down -v` and what was left behind. Also `build --no-cache` with stdout and stderr from a `RUN` step, and `events --json` across a restart. After normalising ids, times and paths the outputs are the same. What differs is the order of parallel progress rows and of the orphan container list, and that order also changes between two runs of develop. - Each new test fails when its control is removed: - `DrivenBody`: returning the connection error first, returning `Pending` while the connection is pending, and ending the body when the connection finishes cleanly each fail their own test. - `ConnState`: recording no completion polls a finished fake future again and fails two tests. - `logs -f` on a container that prints its first line 3 s after start: the line must arrive within 2 s of being printed. With a 2.5 s delay added to every frame `DrivenBody` returns, the line took 2.5 s and the test failed. - `logs -f` whose reader closes the pipe after one line must exit 0 within 5 s. With `stop_on_write_error` ignoring `BrokenPipe`, podup was still running after 5 s. Not measured: Windows (named pipe) and macOS (`podman machine`) against a real Podman. The streaming path is the same code on every transport. The streaming methods of the library now return `Response<DrivenBody>` instead of `Response<Incoming>`. The library is internal code (#1923); the command line does not change. Closes #1900 Signed-off-by: Jaro-c <75870284+Jaro-c@users.noreply.github.com>
chore(release): bump to 5.10.2 Bumps `Cargo.toml`, `Cargo.lock` and `debian/changelog` to 5.10.2 together. Patch release carrying #1927 (the streaming libpod connection read in the same task as the body, which takes `podup logs` from about 20.5 s to 12.3 s on a 2.1 M-line log with the same output), shipped on its own so anything it changes can be traced to it. Signed-off-by: Jaro-c <75870284+Jaro-c@users.noreply.github.com>
Jaro-c
added
prio:P2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Release 5.10.2: the streaming libpod connection read in the same task as the body, so
podup logstakes about 40% less time on a long log with the same output (#1927). Also carries the documentation that describes the library target as internal code (#1923).Signed-off-by: Jaro-c 75870284+Jaro-c@users.noreply.github.com