Skip to content

Deepen validation range - #5

Open
Mshehu5 wants to merge 2 commits into
commit-validation-gixfrom
deepen-validation-range
Open

Mshehu5 wants to merge 2 commits into
commit-validation-gixfrom
deepen-validation-range

Conversation

@Mshehu5

@Mshehu5 Mshehu5 commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Closes #2.

actions/checkout makes a shallow clone by default, so in CI the base
of BASE..HEAD is often missing. Today CI gets around this by cloning
the full history (fetch-depth: 0). This PR makes the tool fetch only
the history it needs.

  • Commit 1 accepts only X, ^X and A..B, in every clone.
    X^!, X^@, A...B and --all need parents or shared history
    that a shallow clone may not have, and nothing uses them today.
  • Commit 2 fetches the base commit if it's missing, then fetches
    more history (8, 16, 32 commits…) until the result can't change. It
    uses git fetch, which already handles CI login, and stops with an
    error if origin can't provide more.

@Mshehu5 Mshehu5 mentioned this pull request Oct 14, 2026
@Mshehu5
Mshehu5 changed the base branch from main to commit-validation October 14, 2026 10:39
@Mshehu5
Mshehu5 added this pull request to stack #6 October 14, 2026 10:40
@Mshehu5
Mshehu5 force-pushed the deepen-validation-range branch from 0314f97 to 39de370 Compare October 14, 2026 10:44
@yuval-block

Copy link
Copy Markdown
Collaborator

looks like this needs rebase it duplicates some commits that are in #5 with different commit IDs

@Mshehu5
Mshehu5 force-pushed the deepen-validation-range branch from 39de370 to c96c6ea Compare October 14, 2026 18:20
@oakagent

Copy link
Copy Markdown

ACK c96c6ea (automated): reviewed 1 commit c96c6ea; no findings.

@Mshehu5
Mshehu5 force-pushed the deepen-validation-range branch 2 times, most recently from 9a4c7b7 to 504bbaf Compare October 15, 2026 18:55
@Mshehu5
Mshehu5 force-pushed the deepen-validation-range branch from 504bbaf to 3724395 Compare October 15, 2026 19:10
@oakagent

Copy link
Copy Markdown

ACK 3724395 (automated): reviewed 1 commit 3724395; no findings.

@bc1cindy bc1cindy left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

can you please update the PR description?

Comment thread src/history.rs Outdated
loop {
// Reopen after fetching so cached commits use the new shallow boundaries.
let repo = Repository::open(repo.path()).map_err(Error::DiscoverRepository)?;
// Even after fetching a missing SHA, one commit needs no parent history.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment is confusing, we don't fetch "a missing SHA"

also for diff injection i think we do need the parent of the single revision as well?

Comment thread src/history.rs Outdated
}

// Fetch commits or more ancestry without changing branches or staged files.
fn fetch(repo: &Repository, tips: &[Oid], deepen: Option<u32>) -> Result<(), Error> {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't this be done via the API? feels very strange to mix programmatic access with shell based access to the git repo

invoking system commands also tends to complicated error handling etc and is more sensitive to the environment so unless this functionality is unavailable i think it's best to avoid invoking the CLI

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can
It’s been a while, but one reason I kept the CLI was that FetchOptions::depth() sets an absolute depth, while git fetch --deepen=N extends the existing shallow boundary. Our incremental fetching relies on that distinction to preserve existing history.

The git2 approach also needed additional authentication and HTTP-header configuration handling for private repositories. The Git command already uses the existing configuration, so it avoided adding more code to maintain and review. git2 supports fetching and authentication the concern was preserving the behavior without introducing unnecessary complexity

@Mshehu5
Mshehu5 force-pushed the deepen-validation-range branch from 3724395 to 52eb349 Compare November 5, 2026 18:14
@Mshehu5
Mshehu5 force-pushed the deepen-validation-range branch from 52eb349 to a9af3a1 Compare November 5, 2026 22:30
@Mshehu5
Mshehu5 force-pushed the deepen-validation-range branch from a9af3a1 to 26024bc Compare November 5, 2026 22:42
Comment thread src/history.rs Outdated
let commit = resolve(repo, revision)?;
selection.tips.push(commit.id());
if truncated.contains(&commit.id()) {
selection.needs_parents.push(commit.id());

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Redundant HEAD makes HEAD^! fetch at a shallow boundary

When HEAD is the shallow boundary, HEAD^! HEAD selects only HEAD, but the extra plain HEAD revision prevents the all-^! fast path, so the HEAD^! branch queues HEAD as needing its missing parents at line 131 and fetches. With an unreachable origin the command fails even though HEAD^! alone succeeds without fetching.

oak · automated · 26024bc

Base automatically changed from commit-validation to main November 6, 2026 17:19
@Mshehu5
Mshehu5 force-pushed the deepen-validation-range branch from 26024bc to 89d5d0a Compare November 6, 2026 17:19
Comment thread src/history.rs Outdated
let depth_flag = match deepen {
Some(count) => {
eprintln!("Deepening history by {count} commits");
format!("--deepen={count}")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dangling shallow commit fails to deepen on older git servers

A single revision naming a commit unreachable from an advertised ref is fetched at the shallow cutoff, and the deepen loop sends that SHA as the want; on origins whose git predates relative-deepen handling of unreachable shallow commits, upload-pack ignores it, so the boundary remains and the command errors with NoHistoryProgress even though origin has the parent. This requires an origin that serves unreachable SHAs, such as one with uploadpack.allowAnySHA1InWant.

oak · automated · 89d5d0a

@Mshehu5
Mshehu5 force-pushed the deepen-validation-range branch from 89d5d0a to 0e380c1 Compare November 13, 2026 14:09
@Mshehu5
Mshehu5 removed this pull request from stack #6 November 13, 2026 14:17
@Mshehu5
Mshehu5 changed the base branch from main to commit-validation-gix November 13, 2026 14:17
@Mshehu5
Mshehu5 added this pull request to stack #13 November 13, 2026 14:17
X^!, X^@, A...B and --all need parents, common ancestors or every ref.
A shallow clone may not have downloaded those, so the next commit
could not deepen for them. Rejecting them in every clone keeps a
command's behaviour the same in CI and locally, and leaves one
revision parser for both kinds of clone. Nothing uses the removed
forms today.
CI usually checks out a shallow clone, where the base of a range and
the history behind it may be missing. Fetch a missing base commit by
its ID, then deepen from the cut-off points behind the selected
revisions until missing history can no longer change which commits
are selected.

Fetching runs the git command: it deepens relative to the current
cut-off, works with local remotes, and uses the credentials
actions/checkout stores in http.extraheader, none of which libgit2
supports. If origin cannot supply more history, stop with an error
instead of fetching forever.
@Mshehu5
Mshehu5 force-pushed the deepen-validation-range branch from 0e380c1 to 3ae8dfe Compare November 13, 2026 15:39
Comment thread src/history.rs
}
// A selected commit's parents are missing, so older selected commits
// may be missing too.
if selected.iter().any(|id| cutoffs.contains(id)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certain shallow selection fails with NoHistoryProgress

complete treats a selected shallow boundary that records a parent as incomplete even when all those parents are already excluded, so the missing history cannot change the selection. In a shallow clone from a shallow origin, base..HEAD with base as HEAD's parent fails with NoHistoryProgress instead of returning [HEAD].

Test that fails on this commit
    #[test]
    fn shallow_boundary_with_excluded_parent_is_complete() {
        let remote = Remote::new();
        let commits = remote.chain(None, 2, "Main");
        let (base, head) = (commits[0], commits[1]);
        let origin = remote.clone_shallow(head, 1);
        let dir = origin.workdir().unwrap().to_owned();
        git(&dir, &["fetch", "-q", "--depth=1", "origin", &base.to_string()]);
        git(&dir, &["config", "uploadpack.allowAnySHA1InWant", "true"]);
        let root = tempfile::tempdir().unwrap();
        let url = format!("file://{}", dir.display());
        git(root.path(), &["clone", "-q", "--depth=1", &url, "clone"]);
        let clone = gix::open(root.path().join("clone")).unwrap();
        let commits = select_in(&clone, &[&format!("{base}..HEAD")]).unwrap();
        assert_eq!(commits, vec![head]);
    }

cargo test --lib shallow_boundary_with_excluded_parent_is_complete

oak · automated · 3ae8dfe

Comment thread src/history.rs
}
}
for input in revisions {
let spec = repo.rev_parse(*input).context(Error::ParseRevision)?;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shallow clone lacking HEAD~2 returns parse error

In a shallow clone that lacks HEAD~2, HEAD~2...HEAD fails with ParseRevision instead of UnsupportedRevision because repo.rev_parse resolves the whole revision before the match, although the comment promises consistent rejection.

Rejecting them everywhere keeps full and shallow clones consistent.

— fungi-protocol/validate-commits@126ad0b · src/history.rs#L26

oak · automated · 3ae8dfe

Comment thread src/shallow.rs
// are never fetched.
pub(crate) fn missing_commit(repo: &Repository, revision: &str) -> Option<ObjectId> {
// SHA-1 and SHA-256 repositories use IDs of different lengths.
if revision.len() != repo.object_hash().len_in_hex() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depth-1 clone cannot fetch HEAD^ parent

In a depth-1 clone, HEAD^..HEAD resolves to a parent object that is absent; missing_commit only accepts full hex IDs, so parse_revisions fails with PeelToCommit instead of fetching. The comment documents A..B as supported.

// Accept only X, ^X and A..B. Other forms (X^!, X^@, A...B) need parents // or common ancestors, which a shallow clone may not have downloaded.

— fungi-protocol/validate-commits@3ae8dfe · src/history.rs#L32-L33

oak · automated · 3ae8dfe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

deepen as necessary to validate commit range

4 participants