Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Whole-upstream tutorial** - Added a second tutorial showing how to vendor an entire upstream repository into a local directory with `remote:.` or `remote:/`.
- **Claude sandbox starter** - Added `sbx-kits/claude` so repo-local sandbox examples now cover both OpenCode and Claude.
- **Private fork migration tutorial** - Added a guarded tutorial for migrating an existing private fork or derivative repo to a repo-root `git-cross` patch with explicit backup and override review steps.
- **Repo split tutorial** - Added `docs/tutorials/split-repo-into-upstream-and-private.md`, a concise walkthrough for splitting one original repo into a clean secret-free upstream plus a private `git-cross`-managed fork. Uses a single shared rule: one `.crossignore` and one `git ls-files --ignored --exclude-from=.crossignore` selection piped to `git rm` (remove from upstream) or `rsync` (back up and keep), so the upstream and the fork can never disagree about what is private. The selection command is documented as a repeatable preview (refine `.crossignore` in a loop until it lists exactly the private files) and clarified as non-destructive — the seed removal only strips the throwaway upstream copy while private files stay in the fork as git-cross overlays. The upstream-seed removal deletes the tracked-only selection (`--cached`, without `--others`) with `rm -rf` followed by `git add -A`, instead of `git rm`: `git rm` aborts the whole batch on untracked/gitignored pathspecs (`node_modules/`, `.envrc.local`) and on submodule/embedded-repo gitlinks (`could not lookup name for submodule ...`), whereas working-tree deletion + restage handles files, directories, and gitlinks uniformly. Documented that `rm` has no `--dry-run` (preview via `xargs … echo` or confirm via `rm -rI`/`-ri` with `xargs -o`), and that `test/` matches a `test` directory at any depth while `*/test/*` does not. The backup keeps `--others` so untracked private files are still copied. Each step is labelled with the repo it runs in.

### Changed
- **`.crossignore` entry syntax** - Override review behavior now uses plain non-comment `.crossignore` lines. Supported forms include basename entries such as `.env` anywhere under the patch, basename globs such as `*.env`, and directory entries such as `config/`. Full `.gitignore` semantics are still not supported.
- **README workflow guidance** - Documented the current `.crossignore` behavior as a review-oriented workflow for local overlay files layered on top of upstream-managed content.

### Fixed
- **Migration tutorial backup command** - Removed the broken recommendation to call the internal Just `_crossignore_overrides` recipe during private-fork migration. The backup step now expands `.crossignore` (which is a gitignore-style *pattern* prescription, not a literal file list) against the real working tree with `git ls-files --ignored --exclude-from=.crossignore` piped into `rsync`, so globs and not-yet-present paths no longer cause `rsync --files-from` "No such file or directory" failures. Documented the safer "clean upstream seed repo first, then patch into the private repo" workflow.
- **Rust override diff build failure** - Renamed a Rust loop variable that used the reserved keyword `override`, restoring Rust CLI builds and regression execution on current toolchains.
- **Just override warning output** - Removed a shell-breaking semicolon from the override review warning so `just cross diff` prints the manual review commands correctly.
- **P0: Sparse checkout broken on newer Git versions** - `git sparse-checkout set <path>` in `--no-cone` mode no longer reliably checks out directories without trailing `/`. Fixed across all three implementations (Go, Rust, Justfile.cross) by appending `/` to sparse-checkout patterns and using `git read-tree -mu HEAD` instead of bare `git checkout` (which can no-op after `--no-checkout`).
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,7 @@ This lets your main repo keep its local opinionated result while the upstream co
- [Tutorial: Local Overlays And Upstream Contribution](docs/tutorials/local-overlays-and-upstream.md)
- [Tutorial: Whole Upstream Into A Local Directory](docs/tutorials/whole-upstream-into-local-dir.md)
- [Tutorial: Migrate A Private Fork To git-cross](docs/tutorials/migrating-private-fork-to-git-cross.md)
- [Tutorial: Split One Repo Into A Clean Upstream And A Private Overlay](docs/tutorials/split-repo-into-upstream-and-private.md)
- [Sandbox Kits](sbx-kits/README.md)

## Commands
Expand Down
2 changes: 2 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@

## Completed Tasks

- [x] Fix the private-fork migration tutorial so it no longer points users at the internal `_crossignore_overrides` Just recipe, and document the cleaner "seed a public upstream, then track it from the private repo" migration path.
- [x] Add a standalone tutorial (`docs/tutorials/split-repo-into-upstream-and-private.md`) for splitting one mixed private repo into a clean upstream plus a private git-cross-managed overlay, cross-linked from README and the migration tutorial.
- [x] Refactor remaining Rust commands to use `duct` for better error visibility.
- [x] Complete `push` command verification in native implementations.
- [x] Integrate integration tests (001-009) into a unified test runner.
Expand Down
49 changes: 49 additions & 0 deletions docs/tutorials/migrating-private-fork-to-git-cross.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,53 @@ Assume:
- your private repo also contains local-only files such as `.env`, `.env.local`, `docker-compose.override.yml`, or private config directories
- you want the repo to stay at repo root, not move upstream content into `vendor/...`

## Before Step 1: Decide Where The Upstream Comes From

There are two valid starting points.

### Case A: A Clean Upstream Repo Already Exists

If you already have a real upstream repository that contains only the shareable code, continue with Step 1.

### Case B: Your Current Private Repo Is The Only Copy

If the current private repo is the only place where the code exists and it mixes:

- upstream-worthy project files
- private files such as `.env`
- machine-local or company-local overrides

create a clean upstream seed repository first, then come back and migrate the private repo with `git-cross`.

This split has its own focused walkthrough:
`docs/tutorials/split-repo-into-upstream-and-private.md`.

The core idea there is a single shared rule: one `.crossignore` lists the private
patterns, and `git ls-files` selects the matching files for both jobs —

```bash
# back up / keep (rsync): include untracked too
git ls-files -z --cached --others --ignored --exclude-from=.crossignore

# remove from upstream seed (delete from copy): tracked only
git ls-files -z --cached --ignored --exclude-from=.crossignore
```

- the tracked list, deleted with `rm -rf` then `git add -A`, strips those files from the new upstream seed
- the full list piped to `rsync` in your private repo *backs up and keeps* them

Use `rm` + `git add -A` rather than `git rm` for the seed: `git rm` aborts the
whole batch on untracked pathspecs or on a submodule it cannot name. Deleting from
the working tree and restaging handles files, directories, and submodule gitlinks
uniformly.

Do the split there, then return here at Step 1 with a clean upstream in hand.

Important note:

- if secrets or private files were ever committed into Git history and you plan to publish that history, do a real history rewrite first
- the split tutorial only covers the working-tree split, not secret-history cleanup

## Step 1: Make A Safety Snapshot

Before changing anything, create a backup branch or tag.
Expand Down Expand Up @@ -132,6 +179,8 @@ If you want upstream contribution later, the cleanest pattern is to register a w

If you only want to mirror the original upstream first, register the original upstream now and switch to a fork later.

If you created a clean upstream seed repository in the earlier split step, register that repository here.

Example:

```bash
Expand Down
187 changes: 187 additions & 0 deletions docs/tutorials/split-repo-into-upstream-and-private.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,187 @@
# Tutorial: Split One Repo Into A Clean Upstream And A Private Overlay

Start from one **original repo** that mixes shareable code with secrets, and end
with two repos:

| Role | Path | What it holds |
|------|------|---------------|
| **original repo** | `project/` | where you start: everything mixed together |
| **upstream repo** | `../project-upstream/` | new, clean, publishable, no secrets |
| **private fork** | `project/` (same dir as original) | the original repo, now tracking upstream via git-cross + local overlays |

The original repo *becomes* the private fork in place. The upstream repo is a new
clean copy you publish. Each step below marks which repo you run it in.

Use this when the original repo is the only copy of the code. If a clean upstream
already exists, use `docs/tutorials/migrating-private-fork-to-git-cross.md`.

> Safety: this only splits the working tree, not Git history. If secrets were
> ever committed and you will publish the full history, rewrite it first with
> `git filter-repo` or BFG. Seeding a fresh snapshot repo (below) avoids that —
> the new upstream starts from one clean commit.

## 1. Snapshot the original repo · *run in: original repo*

```bash
git checkout -b backup/pre-split && git push origin backup/pre-split
```

## 2. Write `.crossignore` · *run in: original repo*

This one file is the single source of truth. It drives what the upstream seed
drops, what you back up, and the overlay markers — so the sets cannot drift.

```bash
cat > .crossignore <<'EOF'
.env*
*.env
docker-compose.override.yml
config/
test/
EOF
```

Patterns are gitignore-style. Match what you mean:

- `.env` is only the literal file; `.env*` also catches `.env.local` / `.env.secrets`
- `config/` is the whole `config` tree
- `test/` matches a directory named `test` **at any depth** (recursive). Do not
use `*/test/*` — `*` never crosses `/`, so that only matches `test` one level
deep and only its immediate children

### The shared rule

Both jobs select files from the same `.crossignore` with `git ls-files`. The only
difference is one flag, because the two destinations handle different file sets:

```bash
# back up (rsync): include untracked files too — rsync copies whatever exists
git ls-files -z --cached --others --ignored --exclude-from=.crossignore

# remove from upstream (delete from copy): tracked files only
git ls-files -z --cached --ignored --exclude-from=.crossignore
```

The upstream side drops `--others`: untracked or gitignored files
(`node_modules/`, `.envrc.local`, …) are never part of the upstream commit
anyway, so they need no removal.

The upstream side also deletes with `rm`, **not** `git rm`. `git rm` aborts the
whole command on anything it cannot resolve — an untracked pathspec
(`pathspec ... did not match any files`) or a submodule / embedded repo
(`could not lookup name for submodule ...`). Deleting from the working tree and
re-running `git add -A` sidesteps both and treats files, directories, and
submodule gitlinks uniformly.

**It is a preview, so refine it in a loop.** Run the command on its own first,
read the output, edit `.crossignore`, run again — repeat until it lists exactly
the files you want to keep private. Only then pipe it anywhere.

**Nothing is deleted from your repo.** The deletion in step 3 only strips the
*throwaway upstream copy*; your originals never leave the private fork. The split
just sorts files by role: private files stay in the fork, shareable files go
upstream.

## 3. Create the upstream repo · *original repo → upstream repo*

Copy the tree out, then delete the tracked files matched by `.crossignore` so
exactly the private files are dropped.

```bash
# from the original repo
mkdir -p ../project-upstream
rsync -av --exclude .git ./ ../project-upstream/

# now work in the upstream repo
cd ../project-upstream
git init -b main
git add -A

# preview: refine .crossignore until this lists exactly the private files
git ls-files --cached --ignored --exclude-from=.crossignore

# delete the matched paths (files, dirs, and submodule gitlinks), then restage
git ls-files -z --cached --ignored --exclude-from=.crossignore | xargs -0 -r rm -rf --
git add -A

rm -f .crossignore # overlay marker does not belong upstream
git add -A
git commit -m "Initial upstream seed"
git remote add origin git@github.com:your-org/project.git
git push -u origin main

cd - # back to the original repo
```

`rm` has no `--dry-run`. To dry-run the destructive line, print the command
instead of running it, or ask for confirmation:

```bash
# print what would run, run nothing
git ls-files -z --cached --ignored --exclude-from=.crossignore | xargs -0 -r echo rm -rf --

# delete with a confirmation prompt (Linux rm -rI = one prompt; macOS rm -ri = per file)
# xargs -o reopens the terminal so the prompt can read your keypress; drop -f or it skips the prompt
git ls-files -z --cached --ignored --exclude-from=.crossignore | xargs -0 -r -o rm -ri --
```

Verify no secrets survived before pushing:

```bash
# inside ../project-upstream, before pushing
git ls-files | grep -Ei 'secret|\.env' || echo "no obvious secrets tracked"
```

If a private file is still tracked, your pattern missed it — fix `.crossignore`
(e.g. `*.env*` to also catch `.env.secrets`) and re-run the preview + delete.

## 4. Back up the private files · *run in: original repo*

Same `.crossignore`, this time with `--others` added so untracked private files
are backed up too, piped to `rsync` (only existing files are copied, so globs and
missing paths cause no error):

```bash
mkdir -p ../private-overrides-backup
git ls-files -z --cached --others --ignored --exclude-from=.crossignore \
| rsync -av --from0 --files-from=- ./ ../private-overrides-backup/
```

## 5. Turn the original repo into the private fork · *run in: original repo*

This is the transition: the original repo now tracks the upstream repo via
git-cross.

```bash
git cross use upstream git@github.com:your-org/project.git
git cross patch upstream:. . # `upstream:/` is equivalent
```

Register a writable fork instead if you plan to contribute back.

## 6. Restore overlays and review · *run in: private fork*

```bash
rsync -av ../private-overrides-backup/ ./ 2>/dev/null || true
cat .crossignore # confirm it survived the patch
git cross status # shows `Override` for the root patch
git cross diff .
```

## 7. Commit and day-2 workflow · *run in: private fork*

```bash
git add Crossfile .crossignore . && git commit -m "Track upstream via git-cross"
git push origin main
```

- pull upstream: `git cross sync .`
- review: `git cross diff .` / `git cross status`
- contribute upstream: `git cross push . --message "..."` after confirming no
private files are included

## Related

- `docs/tutorials/migrating-private-fork-to-git-cross.md` — clean upstream already exists
- `docs/tutorials/whole-upstream-into-local-dir.md` — vendor into `vendor/...` instead of root
- `docs/tutorials/local-overlays-and-upstream.md` — overlay a single subdirectory
Loading