FAILED: Rebase shears/seen (#24684776045)#139
Draft
gitforwindowshelper[bot] wants to merge 254 commits intobase/shears/seen-24684776045from
Draft
FAILED: Rebase shears/seen (#24684776045)#139gitforwindowshelper[bot] wants to merge 254 commits intobase/shears/seen-24684776045from
gitforwindowshelper[bot] wants to merge 254 commits intobase/shears/seen-24684776045from
Conversation
[PT: ensure we add an additional element to the argv array] Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This merges the fix for CVE-2026-32631 into the v2.53.x release branch. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When a Unix socket is initialized, the current directory's path is stored so that the cleanup code can `chdir()` back to where it was before exit. If the path that needs to be stored exceeds the default size of the `sun_path` attribute of `struct sockaddr_un` (which is defined as a 108-sized byte array on Linux), a larger buffer needs to be allocated so that it can hold the path, and it is the responsibility of the `unix_sockaddr_cleanup()` function to release that allocated memory. In Git's CI, this stack allocation is not necessary because the code is checked out to `/home/runner/work/git/git`. Concatenate the path `t/trash directory.t0301-credential-cache/.cache/git/credential/socket` and a terminating NUL, and you end up with 96 bytes, 12 shy of the default `sun_path` size. However, I use worktrees with slightly longer paths: `/home/me/projects/git/yes/i/nest/worktrees/to/organize/them/` is more in line with what I have. When I recently tried to locally reproduce a failure of the `linux-leaks` CI job, this t0301 test failed (where it had not failed in CI). The reason: When `credential-cache` tries to reach its daemon initially by calling `unix_sockaddr_init()`, it is expected that the daemon cannot be reached (the idea is to spin up the daemon in that case and try again). However, when this first call to `unix_sockaddr_init()` fails, the code returns early from the `unix_stream_connect()` function _without_ giving the cleanup code a chance to run, skipping the deallocation of above-mentioned path. The fix is easy: do not return early but instead go directly to the cleanup code. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Sverre Rabbelier <srabbelier@gmail.com>
In some implementations, `regexec_buf()` assumes that it is fed lines; Without `REG_NOTEOL` it thinks the end of the buffer is the end of a line. Which makes sense, but trips up this case because we are not feeding lines, but rather a whole buffer. So the final newline is not the start of an empty line, but the true end of the buffer. This causes an interesting bug: $ echo content >file.txt $ git grep --no-index -n '^$' file.txt file.txt:2: This bug is fixed by making the end of the buffer consistently the end of the final line. The patch was applied from https://lore.kernel.org/git/20250113062601.GD767856@coredump.intra.peff.net/ Reported-by: Olly Betts <olly@survex.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
After importing anything with fast-import, we should always let the garbage collector do its job, since the objects are written to disk inefficiently. This brings down an initial import of http://selenic.com/hg from about 230 megabytes to about 14. In the future, we may want to make this configurable on a per-remote basis, or maybe teach fast-import about it in the first place. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
These fixes have been sent to the Git mailing list but have not been picked up by the Git project yet. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows, there are several categories of absolute paths. One such category starts with a backslash and is implicitly relative to the drive associated with the current working directory. Example: c: git clone https://github.com/git-for-windows/git \G4W should clone into C:\G4W. Back in 2017, Juan Carlos Arevalo Baeza reported a bug in Git's handling of those absolute paths was identified, and fixed. Let's make sure that it stays fixed. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When asking the mingw-w64 variant of GCC to compile C11 code, it seems to link implicitly to libwinpthread, which does implement a pthread emulation (that is more complete than Git's). In preparation for vendoring in mimalloc (which requires C11 support), let's keep preferring Git's own pthread emulation. To avoid linker errors where it thinks that the `pthread_self` and the `pthread_create` symbols are defined twice, let's give our version a `win32_` prefix, just like we already do for `pthread_join()`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We are about to vendor in `mimalloc`'s source code which we will want to include `compat/posix.h` after defining that constant. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Update to newer mimalloc versions like this:
update_mimalloc ()
{
test $# = 1 || {
echo "Need a mimalloc version" 1>&2;
return 1
};
for oneline in 'mimalloc: adjust for building inside Git' 'Import the source code of mimalloc';
do
git revert -n HEAD^{/^"$oneline"} && git checkout HEAD -- Makefile && git commit -sm "Temporarily revert \"$oneline\"" -m 'In preparation for upgrading to a newer mimalloc version.' || return 1;
done;
for file in $(git show --format='%n' --name-only --diff-filter=A HEAD^{/^"Import the source code of mimalloc "}) compat/mimalloc/arena-abandon.c compat/mimalloc/free.c compat/mimalloc/libc.c compat/mimalloc/prim/prim.c compat/mimalloc/mimalloc-stats.h;
do
file2=${file#compat/mimalloc/};
case "$file2" in
segment-cache.c)
: no longer needed;
continue
;;
bitmap.h | *.c)
file2=src/$file2
;;
*.h)
file2=include/$file2
;;
esac;
mkdir -p "${file%/*}" && git -C /usr/src/mimalloc/ show "$1":$file2 > "$file" && git add "$file" || {
echo "Failed: $file2 -> $file" 1>&2;
return 1
};
done;
conv_sed='sed -n "/^ *eval/d;/ /p"' && git commit -sm "Import the source code of mimalloc $1" -m "Update to newer mimalloc versions like this:" -m "$(set | sed -n '/^update_mimalloc *() *$/,/^}/{s/^./ &/;p}')" -m ' update_mimalloc $MIMALLOC_VERSION' -m 'For convenience, you can set `MIMALLOC_VERSION` and then run:' -m ' eval "$(git show -s <this-commit> | '"$conv_sed"')"' || return 1;
git cherry-pick HEAD^{/^'mimalloc: adjust for building inside Git'} || return 1
}
update_mimalloc $MIMALLOC_VERSION
For convenience, you can set `MIMALLOC_VERSION` and then run:
eval "$(git show -s <this-commit> | sed -n "/^ *eval/d;/ /p")"
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We want to compile mimalloc's source code as part of Git, rather than requiring the code to be built as an external library: mimalloc uses a CMake-based build, which is not necessarily easy to integrate into the flavors of Git for Windows (which will be the main benefitting port). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Matthias Aßhauer <mha1993@live.de>
It seems to be not exactly rare on Windows to install NTFS junction points (the equivalent of "bind mounts" on Linux/Unix) in worktrees, e.g. to map some development tools into a subdirectory. In such a scenario, it is pretty horrible if `git clean -dfx` traverses into the mapped directory and starts to "clean up". Let's just not do that. Let's make sure before we traverse into a directory that it is not a mount point (or junction). This addresses git-for-windows#607 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
By defining `USE_MIMALLOC`, Git can now be compiled with that nicely-fast and small allocator. Note that we have to disable a couple `DEVELOPER` options to build mimalloc's source code, as it makes heavy use of declarations after statements, among other things that disagree with Git's conventions. We even have to silence some GCC warnings in non-DEVELOPER mode. For example, the `-Wno-array-bounds` flag is needed because in `-O2` builds, trying to call `NtCurrentTeb()` (which `_mi_thread_id()` does on Windows) causes the bogus warning about a system header, likely related to https://sourceforge.net/p/mingw-w64/mailman/message/37674519/ and to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99578: C:/git-sdk-64-minimal/mingw64/include/psdk_inc/intrin-impl.h:838:1: error: array subscript 0 is outside array bounds of 'long long unsigned int[0]' [-Werror=array-bounds] 838 | __buildreadseg(__readgsqword, unsigned __int64, "gs", "q") | ^~~~~~~~~~~~~~ Also: The `mimalloc` library uses C11-style atomics, therefore we must require that standard when compiling with GCC if we want to use `mimalloc` (instead of requiring "only" C99). This is what we do in the CMake definition already, therefore this commit does not need to touch `contrib/buildsystems/`. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
NTFS junctions are somewhat similar in spirit to Unix bind mounts: they point to a different directory and are resolved by the filesystem driver. As such, they appear to `lstat()` as if they are directories, not as if they are symbolic links. _Any_ user can create junctions, while symbolic links can only be created by non-administrators in Developer Mode on Windows 10. Hence NTFS junctions are much more common "in the wild" than NTFS symbolic links. It was reported in git-for-windows#2481 that adding files via an absolute path that traverses an NTFS junction: since 1e64d18 (mingw: do resolve symlinks in `getcwd()`), we resolve not only symbolic links but also NTFS junctions when determining the absolute path of the current directory. The same is not true for `git add <file>`, where symbolic links are resolved in `<file>`, but not NTFS junctions. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Windows' equivalent to "bind mounts", NTFS junction points, can be unlinked without affecting the mount target. This is clearly what users expect to happen when they call `git clean -dfx` in a worktree that contains NTFS junction points: the junction should be removed, and the target directory of said junction should be left alone (unless it is inside the worktree). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
While Git for Windows does not _ship_ Python (in order to save on bandwidth), MSYS2 provides very fine Python interpreters that users can easily take advantage of, by using Git for Windows within its SDK. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Thorough benchmarking with repacking a subset of linux.git (the commit history reachable from 93a6fef ([PATCH] fix the SYSCTL=n compilation, 2007-02-28), to be precise) suggest that this allocator is on par, in multi-threaded situations maybe even better than nedmalloc: `git repack -adfq` with mimalloc, 8 threads: 31.166991900 27.576763800 28.712311000 27.373859000 27.163141900 `git repack -adfq` with nedmalloc, 8 threads: 31.915032900 27.149883100 28.244933700 27.240188800 28.580849500 In a different test using GitHub Actions build agents (probably single-threaded, a core-strength of nedmalloc)): `git repack -q -d -l -A --unpack-unreachable=2.weeks.ago` with mimalloc: 943.426 978.500 939.709 959.811 954.605 `git repack -q -d -l -A --unpack-unreachable=2.weeks.ago` with nedmalloc: 995.383 952.179 943.253 963.043 980.468 While these measurements were not executed with complete scientific rigor, as no hardware was set aside specifically for these benchmarks, it shows that mimalloc and nedmalloc perform almost the same, nedmalloc with a bit higher variance and also slightly higher average (further testing suggests that nedmalloc performs worse in multi-threaded situations than in single-threaded ones). In short: mimalloc seems to be slightly better suited for our purposes than nedmalloc. Seeing that mimalloc is developed actively, while nedmalloc ceased to see any updates in eight years, let's use mimalloc on Windows instead. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Since commit 0c499ea (send-pack: demultiplex a sideband stream with status data, 2010-02-05) the send-pack builtin uses the side-band-64k capability if advertised by the server. Unfortunately this breaks pushing over the dump git protocol if used over a network connection. The detailed reasons for this breakage are (by courtesy of Jeff Preshing, quoted from https://groups.google.com/d/msg/msysgit/at8D7J-h7mw/eaLujILGUWoJ): MinGW wraps Windows sockets in CRT file descriptors in order to mimic the functionality of POSIX sockets. This causes msvcrt.dll to treat sockets as Installable File System (IFS) handles, calling ReadFile, WriteFile, DuplicateHandle and CloseHandle on them. This approach works well in simple cases on recent versions of Windows, but does not support all usage patterns. In particular, using this approach, any attempt to read & write concurrently on the same socket (from one or more processes) will deadlock in a scenario where the read waits for a response from the server which is only invoked after the write. This is what send_pack currently attempts to do in the use_sideband codepath. The new config option `sendpack.sideband` allows to override the side-band-64k capability of the server, and thus makes the dumb git protocol work. Other transportation methods like ssh and http/https still benefit from the sideband channel, therefore the default value of `sendpack.sideband` is still true. Signed-off-by: Thomas Braun <thomas.braun@byte-physics.de> Signed-off-by: Oliver Schneider <oliver@assarbad.net> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In 1e64d18 (mingw: do resolve symlinks in `getcwd()`) a problem was introduced that causes git for Windows to stop working with certain mapped network drives (in particular, drives that are mapped to locations with long path names). Error message was "fatal: Unable to read current working directory: No such file or directory". Present change fixes this issue as discussed in git-for-windows#2480 Signed-off-by: Bjoern Mueller <bjoernm@gmx.de>
Update clink.pl to link with either libcurl.lib or libcurl-d.lib depending on whether DEBUG=1 is set. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Co-authored-by: Dennis Ameling <dennis@dennisameling.com> Signed-off-by: Ian Bearman <ianb@microsoft.com> Signed-off-by: Dennis Ameling <dennis@dennisameling.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
In this context, a "feature" is a dependency combined with its own dependencies. Signed-off-by: Ian Bearman <ianb@microsoft.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Dennis Ameling <dennis@dennisameling.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Some platforms (e.g. Windows) provide API functions to resolve paths much quicker. Let's offer a way to short-cut `strbuf_realpath()` on those platforms. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When we commit the template directory as part of `make vcxproj`, the `branches/` directory is not actually commited, as it is empty. Two tests were not prepared for that situation. This developer tried to get rid of the support for `.git/branches/` a long time ago, but that effort did not bear fruit, so the best we can do is work around in these here tests. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The vcpkg_install batch file depends on the availability of a working Git on the CMD path. This may not be present if the user has selected the 'bash only' option during Git-for-Windows install. Detect and tell the user about their lack of a working Git in the CMD window. Fixes git-for-windows#2348. A separate PR git-for-windows/build-extra#258 now highlights the recommended path setting during install. Signed-off-by: Philip Oakley <philipoakley@iee.email>
There are no Windows/ARM64 agents in GitHub Actions yet, therefore we just skip adjusting the `vs-test` job for now. Signed-off-by: Dennis Ameling <dennis@dennisameling.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
There is a Win32 API function to resolve symbolic links, and we can use that instead of resolving them manually. Even better, this function also resolves NTFS junction points (which are somewhat similar to bind mounts). This fixes git-for-windows#2481. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Windows paths are typically limited to MAX_PATH = 260 characters, even
though the underlying NTFS file system supports paths up to 32,767 chars.
This limitation is also evident in Windows Explorer, cmd.exe and many
other applications (including IDEs).
Particularly annoying is that most Windows APIs return bogus error codes
if a relative path only barely exceeds MAX_PATH in conjunction with the
current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the
infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG.
Many Windows wide char APIs support longer than MAX_PATH paths through the
file namespace prefix ('\\?\' or '\\?\UNC\') followed by an absolute path.
Notable exceptions include functions dealing with executables and the
current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as
well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...).
Introduce a handle_long_path function to check the length of a specified
path properly (and fail with ENAMETOOLONG), and to optionally expand long
paths using the '\\?\' file namespace prefix. Short paths will not be
modified, so we don't need to worry about device names (NUL, CON, AUX).
Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be
limited to MAX_PATH (at least not on Win7), so we can use it to do the
heavy lifting of the conversion (translate '/' to '\', eliminate '.' and
'..', and make an absolute path).
Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH
limit.
Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs
that support long paths.
While improved error checking is always active, long paths support must be
explicitly enabled via 'core.longpaths' option. This is to prevent end
users from shooting themselves in the foot by checking out files that Windows
Explorer, cmd/bash or their favorite IDE cannot handle.
Test suite:
Test the case is when the full pathname length of a dir is close
to 260 (MAX_PATH).
Bug report and an original reproducer by Andrey Rogozhnikov:
msysgit#122 (comment)
[jes: adjusted test number to avoid conflicts, added support for
chdir(), etc]
Thanks-to: Martin W. Kirst <maki@bitkings.de>
Thanks-to: Doug Kelly <dougk.ff7@gmail.com>
Original-test-by: Andrey Rogozhnikov <rogozhnikov.andrey@gmail.com>
Signed-off-by: Karsten Blees <blees@dcon.de>
Signed-off-by: Stepan Kasal <kasal@ucw.cz>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Josh Soref <jsoref@gmail.com>
When trying to ensure that long paths are handled correctly, we first normalize absolute paths as we encounter them. However, if the path is a so-called "drive-less" absolute path, i.e. if it is relative to the current drive but _does_ start with a directory separator, we would want the normalized path to be such a drive-less absolute path, too. Let's do that, being careful to still include the drive prefix when we need to go through the `\\?\` dance (because there, the drive prefix is absolutely required). This fixes git-for-windows#4586. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Update wchar_t buffers to use MAX_LONG_PATH instead of MAX_PATH and call xutftowcs_long_path() in the Win32 backend source files. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com>
On Windows, git repositories may have extra files which need cleaned (e.g., a build directory) that may be arbitrarily deep. Suggest using `core.longPaths` if such situations are encountered. Fixes: git-for-windows#2715 Signed-off-by: Ben Boeckel <mathstuf@gmail.com>
The `git_terminal_prompt()` function expects the terminal window to be attached to a Win32 Console. However, this is not the case with terminal windows other than `cmd.exe`'s, e.g. with MSys2's own `mintty`. Non-cmd terminals such as `mintty` still have to have a Win32 Console to be proper console programs, but have to hide the Win32 Console to be able to provide more flexibility (such as being resizeable not only vertically but also horizontally). By writing to that Win32 Console, `git_terminal_prompt()` manages only to send the prompt to nowhere and to wait for input from a Console to which the user has no access. This commit introduces a function specifically to support `mintty` -- or other terminals that are compatible with MSys2's `/dev/tty` emulation. We use the `TERM` environment variable as an indicator for that: if the value starts with "xterm" (such as `mintty`'s "xterm_256color"), we prefer to let `xterm_prompt()` handle the user interaction. The most prominent user of `git_terminal_prompt()` is certainly `git-remote-https.exe`. It is an interesting use case because both `stdin` and `stdout` are redirected when Git calls said executable, yet it still wants to access the terminal. When running inside a `mintty`, the terminal is not accessible to the `git-remote-https.exe` program, though, because it is a MinGW program and the `mintty` terminal is not backed by a Win32 console. To solve that problem, we simply call out to the shell -- which is an *MSys2* program and can therefore access `/dev/tty`. Helped-by: nalla <nalla@hamal.uberspace.de> Signed-off-by: Karsten Blees <blees@dcon.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Accessing the Windows console through the special CONIN$ / CONOUT$ devices doesn't work properly for non-ASCII usernames an passwords. It also doesn't work for terminal emulators that hide the native console window (such as mintty), and 'TERM=xterm*' is not necessarily a reliable indicator for such terminals. The new shell_prompt() function, on the other hand, works fine for both MSys1 and MSys2, in native console windows as well as mintty, and properly supports Unicode. It just needs bash on the path (for 'read -s', which is bash-specific). On Windows, try to use the shell to read from the terminal. If that fails with ENOENT (i.e. bash was not found), use CONIN/OUT as fallback. Note: To test this, create a UTF-8 credential file with non-ASCII chars, e.g. in git-bash: 'echo url=http://täst.com > cred.txt'. Then in git-cmd, 'git credential fill <cred.txt' works (shell version), while calling git without the git-wrapper (i.e. 'mingw64\bin\git credential fill <cred.txt') mangles non-ASCII chars in both console output and input. Signed-off-by: Karsten Blees <blees@dcon.de>
To support Git Bash running in a MinTTY, we use a dirty trick to access the MSYS2 pseudo terminal: we execute a Bash snippet that accesses /dev/tty. The idea was to fall back to writing to/reading from CONOUT$/CONIN$ if that Bash call failed because Bash was not found. However, we should fall back even in other error conditions, because we have not successfully read the user input. Let's make it so. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Bert Belder <bertbelder@gmail.com>
On Windows, symbolic links actually have a type depending on the target: it can be a file or a directory. In certain circumstances, this poses problems, e.g. when a symbolic link is supposed to point into a submodule that is not checked out, so there is no way for Git to auto-detect the type. To help with that, we will add support over the course of the next commits to specify that symlink type via the Git attributes. This requires an index_state, though, something that Git for Windows' `symlink()` replacement cannot know about because the function signature is defined by the POSIX standard and not ours to change. So let's introduce a helper function to create symbolic links that *does* know about the index_state. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On Windows, symbolic links have a type: a "file symlink" must point at a file, and a "directory symlink" must point at a directory. If the type of symlink does not match its target, it doesn't work. Git does not record the type of symlink in the index or in a tree. On checkout it'll guess the type, which only works if the target exists at the time the symlink is created. This may often not be the case, for example when the link points at a directory inside a submodule. By specifying `symlink=file` or `symlink=dir` the user can specify what type of symlink Git should create, so Git doesn't have to rely on unreliable heuristics. Signed-off-by: Bert Belder <bertbelder@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
To verify that the symlink is resolved correctly, we use the fact that `git.exe` is a native Win32 program, and that `git.exe config -f <path>` therefore uses the native symlink resolution. Signed-off-by: Bert Belder <bertbelder@gmail.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The main idea of this patch is that even if we have to look up the absolute path of the script, if only the basename was specified as argv[0], then we should use that basename on the command line, too, not the absolute path. This patch will also help with the upcoming patch where we automatically substitute "sh ..." by "busybox sh ..." if "sh" is not in the PATH but "busybox" is: we will do that by substituting the actual executable, but still keep prepending "sh" to the command line. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
BusyBox comes with a ton of applets ("applet" being the identical
concept to Git's "builtins"). And similar to Git's builtins, the applets
can be called via `busybox <command>`, or the BusyBox executable can be
copied/hard-linked to the command name.
The similarities do not end here. Just as with Git's builtins, it is
problematic that BusyBox' hard-linked applets cannot easily be put into
a .zip file: .zip archives have no concept of hard-links and therefore
would store identical copies (and also extract identical copies,
"inflating" the archive unnecessarily).
To counteract that issue, MinGit already ships without hard-linked
copies of the builtins, and the plan is to do the same with BusyBox'
applets: simply ship busybox.exe as single executable, without
hard-linked applets.
To accommodate that, Git is being taught by this commit a very special
trick, exploiting the fact that it is possible to call an executable
with a command-line whose argv[0] is different from the executable's
name: when `sh` is to be spawned, and no `sh` is found in the PATH, but
busybox.exe is, use that executable (with unchanged argv).
Likewise, if any executable to be spawned is not on the PATH, but
busybox.exe is found, parse the output of `busybox.exe --help` to find
out what applets are included, and if the command matches an included
applet name, use busybox.exe to execute it.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
It is convenient to assume that everybody who wants to build & test Git has access to a working `iconv` executable (after all, we already pretty much require libiconv). However, that limits esoteric test scenarios such as Git for Windows', where an end user installation has to ship with `iconv` for the sole purpose of being testable. That payload serves no other purpose. So let's just have a test helper (to be able to test Git, the test helpers have to be available, after all) to act as `iconv` replacement. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We already have a directory where we store files intended for use by multiple test scripts. The same directory is a better home for the test-binary-*.png files than t/. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The idea is to allow running the test suite on MinGit with BusyBox installed in /mingw64/bin/sh.exe. In that case, we will want to exclude sort & find (and other Unix utilities) from being bundled. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
BusyBox-w32 is a true Win32 application, i.e. it does not come with a POSIX emulation layer. That also means that it does *not* use the Unix convention of separating the entries in the PATH variable using colons, but semicolons. However, there are also BusyBox ports to Windows which use a POSIX emulation layer such as Cygwin's or MSYS2's runtime, i.e. using colons as PATH separators. As a tell-tale, let's use the presence of semicolons in the PATH variable: on Unix, it is highly unlikely that it contains semicolons, and on Windows (without POSIX emulation), it is virtually guaranteed, as everybody should have both $SYSTEMROOT and $SYSTEMROOT/system32 in their PATH. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Traditionally, Git for Windows' SDK uses Bash as its default shell. However, other Unix shells are available, too. Most notably, the Win32 port of BusyBox comes with `ash` whose `pwd` command already prints Windows paths as Git for Windows wants them, while there is not even a `builtin` command. Therefore, let's be careful not to override `pwd` unless we know that the `builtin` command is available. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The -W option is only understood by MSYS2 Bash's pwd command. We already make sure to override `pwd` by `builtin pwd -W` for MINGW, so let's not double the effort here. This will also help when switching the shell to another one (such as BusyBox' ash) whose pwd does *not* understand the -W option. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When running with BusyBox, we will want to avoid calling executables on the PATH that are implemented in BusyBox itself. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
At some stage, t5003-archive-zip wants to add a file that is not ASCII. To that end, it uses /bin/sh. But that file may actually not exist (it is too easy to forget that not all the world is Unix/Linux...)! Besides, we already have perfectly fine binary files intended for use solely by the tests. So let's use one of them instead. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
While it may seem super convenient to some old Unix hands to simpy require Perl to be available when running the test suite, this is a major hassle on Windows, where we want to verify that Perl is not, actually, required in a NO_PERL build. As a super ugly workaround, we "install" a script into /usr/bin/perl reading like this: #!/bin/sh # We'd much rather avoid requiring Perl altogether when testing # an installed Git. Oh well, that's why we cannot have nice # things. exec c:/git-sdk-64/usr/bin/perl.exe "$@" The problem with that is that BusyBox assumes that the #! line in a script refers to an executable, not to a script. So when it encounters the line #!/usr/bin/perl in t5532's proxy-get-cmd, it barfs. Let's help this situation by simply executing the Perl script with the "interpreter" specified explicitly. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When t5605 tries to verify that files are hardlinked (or that they are not), it uses the `-links` option of the `find` utility. BusyBox' implementation does not support that option, and BusyBox-w32's lstat() does not even report the number of hard links correctly (for performance reasons). So let's just switch to a different method that actually works on Windows. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Git for Windows uses MSYS2's Bash to run the test suite, which comes with benefits but also at a heavy price: on the plus side, MSYS2's POSIX emulation layer allows us to continue pretending that we are on a Unix system, e.g. use Unix paths instead of Windows ones, yet this is bought at a rather noticeable performance penalty. There *are* some more native ports of Unix shells out there, though, most notably BusyBox-w32's ash. These native ports do not use any POSIX emulation layer (or at most a *very* thin one, choosing to avoid features such as fork() that are expensive to emulate on Windows), and they use native Windows paths (usually with forward slashes instead of backslashes, which is perfectly legal in almost all use cases). And here comes the problem: with a $PWD looking like, say, C:/git-sdk-64/usr/src/git/t/trash directory.t5813-proto-disable-ssh Git's test scripts get quite a bit confused, as their assumptions have been shattered. Not only does this path contain a colon (oh no!), it also does not start with a slash. This is a problem e.g. when constructing a URL as t5813 does it: ssh://remote$PWD. Not only is it impossible to separate the "host" from the path with a $PWD as above, even prefixing $PWD by a slash won't work, as /C:/git-sdk-64/... is not a valid path. As a workaround, detect when $PWD does not start with a slash on Windows, and simply strip the drive prefix, using an obscure feature of Windows paths: if an absolute Windows path starts with a slash, it is implicitly prefixed by the drive prefix of the current directory. As we are talking about the current directory here, anyway, that strategy works. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Workflow run failed to rebase shears/seen.
Rebase Summary: seen
From: 2b8a3ab140 (Merge 'readme' into HEAD, 2018-06-07) (81fe37b3dd..2b8a3ab140)
Skipped: 3274ae3 (sideband: mask control characters, 2024-11-06)
Upstream equivalent: 0494953 (sideband: mask control characters, 2026-03-05)
Range-diff
1: 3274ae3 ! 1: 0494953 sideband: mask control characters
@@ Commit message There is likely a need for more fine-grained controls instead of using a "heavy hammer" like this, which will be introduced subsequently. + Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> + Signed-off-by: Junio C Hamano <gitster@pobox.com> ## sideband.c ## @@ sideband.c: void list_config_color_sideband_slots(struct string_list *list, const char *pref @@ sideband.c: void list_config_color_sideband_slots(struct string_list *list, cons +{ + strbuf_grow(dest, n); + for (; n && *src; src++, n--) { -+ if (!iscntrl(*src) || *src == '\t' || *src == '\n') ++ if (!iscntrl(*src) || *src == '\t' || *src == '\n') { + strbuf_addch(dest, *src); -+ else { ++ } else { + strbuf_addch(dest, '^'); -+ strbuf_addch(dest, 0x40 + *src); ++ strbuf_addch(dest, *src == 0x7f ? '?' : 0x40 + *src); + } + } +} @@ t/t5409-colorize-remote-messages.sh: test_expect_success 'fallback to color.ui' + printf "error: Have you \\033[31mread\\033[m this?\\n" >&2 + exec "$@" + EOF -+ test_config_global uploadPack.packObjectshook ./color-me-surprised && ++ test_config_global uploadPack.packObjectsHook ./color-me-surprised && + test_commit need-at-least-one-commit && + git clone --no-local . throw-away 2>stderr && + test_decode_color <stderr >decoded &&Skipped: 0462ca5 (sideband: introduce an "escape hatch" to allow control characters, 2024-11-06)
Upstream equivalent: 9ed1625 (sideband: introduce an "escape hatch" to allow control characters, 2026-03-05)
Range-diff
1: 0462ca5 ! 1: 9ed1625 sideband: introduce an "escape hatch" to allow control characters
@@ Commit message To help with those use cases, give users a way to opt-out of the protections: `sideband.allowControlCharacters`. + Suggested-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> + Signed-off-by: Junio C Hamano <gitster@pobox.com> ## Documentation/config.adoc ## @@ Documentation/config.adoc: include::config/sequencer.adoc[] @@ sideband.c: void list_config_color_sideband_slots(struct string_list *list, cons + strbuf_grow(dest, n); for (; n && *src; src++, n--) { - if (!iscntrl(*src) || *src == '\t' || *src == '\n') + if (!iscntrl(*src) || *src == '\t' || *src == '\n') { ## t/t5409-colorize-remote-messages.sh ## @@ t/t5409-colorize-remote-messages.sh: test_expect_success 'disallow (color) control sequences in sideband' ' EOF - test_config_global uploadPack.packObjectshook ./color-me-surprised && + test_config_global uploadPack.packObjectsHook ./color-me-surprised && test_commit need-at-least-one-commit && + git clone --no-local . throw-away 2>stderr &&Skipped: 8c8e102 (sideband: do allow ANSI color sequences by default, 2024-11-18)
Upstream equivalent: 12f0fda (sideband: do allow ANSI color sequences by default, 2026-03-05)
Range-diff
1: 8c8e102 ! 1: 12f0fda sideband: do allow ANSI color sequences by default
@@ Commit message to the terminal, and `sideband.allowControlCharacters` to override that behavior. - However, some `pre-receive` hooks that are actively used in practice - want to color their messages and therefore rely on the fact that Git - passes them through to the terminal. + However, as reported by brian m. carlson, some `pre-receive` hooks that + are actively used in practice want to color their messages and therefore + rely on the fact that Git passes them through to the terminal, even + though they have no way to determine whether the receiving side can + actually handle Escape sequences (think e.g. about the practice + recommended by Git that third-party applications wishing to use Git + functionality parse the output of Git commands). In contrast to other ANSI escape sequences, it is highly unlikely that coloring sequences can be essential tools in attack vectors that mislead Git users e.g. by hiding crucial information. Therefore we can have both: Continue to allow ANSI coloring sequences to - be passed to the terminal, and neutralize all other ANSI escape - sequences. + be passed to the terminal by default, and neutralize all other ANSI + Escape sequences. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> + Signed-off-by: Junio C Hamano <gitster@pobox.com> ## Documentation/config/sideband.adoc ## @@ @@ Documentation/config/sideband.adoc + this config setting to override this behavior: ++ +-- -+ color:: ++ `default`:: ++ `color`:: + Allow ANSI color sequences, line feeds and horizontal tabs, + but mask all other control characters. This is the default. -+ false:: ++ `false`:: + Mask all control characters other than line feeds and + horizontal tabs. -+ true:: ++ `true`:: + Allow all control characters to be sent to the terminal. +-- @@ sideband.c: static struct keyword_entry keywords[] = { -static int allow_control_characters; +static enum { -+ ALLOW_NO_CONTROL_CHARACTERS = 0, -+ ALLOW_ALL_CONTROL_CHARACTERS = 1, -+ ALLOW_ANSI_COLOR_SEQUENCES = 2 ++ ALLOW_NO_CONTROL_CHARACTERS = 0, ++ ALLOW_ANSI_COLOR_SEQUENCES = 1<<0, ++ ALLOW_DEFAULT_ANSI_SEQUENCES = ALLOW_ANSI_COLOR_SEQUENCES, ++ ALLOW_ALL_CONTROL_CHARACTERS = 1<<1, +} allow_control_characters = ALLOW_ANSI_COLOR_SEQUENCES; /* Returns a color setting (GIT_COLOR_NEVER, etc). */ @@ sideband.c: static enum git_colorbool use_sideband_colors(void) + if (repo_config_get_string_tmp(the_repository, "sideband.allowcontrolcharacters", + &value)) + ; /* huh? `get_maybe_bool()` returned -1 */ ++ else if (!strcmp(value, "default")) ++ allow_control_characters = ALLOW_DEFAULT_ANSI_SEQUENCES; + else if (!strcmp(value, "color")) + allow_control_characters = ALLOW_ANSI_COLOR_SEQUENCES; + else @@ sideband.c: void list_config_color_sideband_slots(struct string_list *list, cons + * Valid ANSI color sequences are of the form + * + * ESC [ [<n> [; <n>]*] m ++ * ++ * These are part of the Select Graphic Rendition sequences which ++ * contain more than just color sequences, for more details see ++ * https://en.wikipedia.org/wiki/ANSI_escape_code#SGR. + */ + + if (allow_control_characters != ALLOW_ANSI_COLOR_SEQUENCES || @@ sideband.c: void list_config_color_sideband_slots(struct string_list *list, cons } @@ sideband.c: static void strbuf_add_sanitized(struct strbuf *dest, const char *src, int n) for (; n && *src; src++, n--) { - if (!iscntrl(*src) || *src == '\t' || *src == '\n') + if (!iscntrl(*src) || *src == '\t' || *src == '\n') { strbuf_addch(dest, *src); -- else { -+ else if ((i = handle_ansi_color_sequence(dest, src, n))) { ++ } else if ((i = handle_ansi_color_sequence(dest, src, n))) { + src += i; + n -= i; -+ } else { + } else { strbuf_addch(dest, '^'); - strbuf_addch(dest, 0x40 + *src); - } + strbuf_addch(dest, *src == 0x7f ? '?' : 0x40 + *src); ## t/t5409-colorize-remote-messages.sh ## @@ t/t5409-colorize-remote-messages.sh: test_expect_success 'fallback to color.ui' ' @@ t/t5409-colorize-remote-messages.sh: test_expect_success 'fallback to color.ui' + printf "error: Have you \\033[31mread\\033[m this?\\a\\n" >&2 exec "$@" EOF - test_config_global uploadPack.packObjectshook ./color-me-surprised && + test_config_global uploadPack.packObjectsHook ./color-me-surprised && @@ t/t5409-colorize-remote-messages.sh: test_expect_success 'disallow (color) control sequences in sideband' ' git clone --no-local . throw-away 2>stderr &&Resolved: 62db9f4 (compat/mingw: handle WSA errors in strerror, 2024-12-22)
kept both u-mingw and u-odb-inmemory entries in alphabetical order in Makefile and t/meson.build
Range-diff
1: 62db9f4 ! 1: d78ccac compat/mingw: handle WSA errors in strerror
Resolved: 3203a5c (t5563: verify that NTLM authentication works, 2025-11-26)
kept both upstream SPNEGO tests and downstream NTLM test before test_done
Range-diff
1: 3203a5c ! 1: 20b09f0 t5563: verify that NTLM authentication works
Dropped (empty after resolution): 7f0f061 (http: extract http_reauth_prepare() from retry paths, 2026-04-13)
kept existing empty_auth_try_negotiate early-return in http_reauth_prepare() from prior Git for Windows patch; REBASE_HEAD's new function body is a subset of what HEAD already has
Resolved: 8699090 (tests: use the correct path separator with BusyBox, 2018-11-19)
kept both upstream's errexit block and patch's PATH_SEP detection block, which were independent additions at the same location
Range-diff
1: 8699090 ! 1: cc362d8 tests: use the correct path separator with BusyBox