Tests

gh-131178: Add tests for `tokenize` command-line interface (#131274) #146734

Job	Run time
Change detection / Create context from changed files	13s
Check if Autoconf files are up to date	32s
Windows MSI${{ '' }}	0s
Docs	0s
Check if generated files are up to date	4m 11s
Address sanitizer (ubuntu-24.04)	11m 56s
Ubuntu SSL tests with OpenSSL (ubuntu-24.04, 3.0.15)	3m 46s
Ubuntu SSL tests with OpenSSL (ubuntu-24.04, 3.1.7)	4m 1s
Cross build Linux	3m 53s
CIFuzz	0s
Ubuntu SSL tests with OpenSSL (ubuntu-24.04, 3.2.3)	4m 14s
Ubuntu SSL tests with OpenSSL (ubuntu-24.04, 3.3.2)	4m 33s
Hypothesis tests on Ubuntu	9m 12s
Ubuntu SSL tests with OpenSSL (ubuntu-24.04, 3.4.0)	4m 47s
WASI / build and test	4m 50s
Windows / build (arm64)	3m 11s
macOS (free-threading) / build and test (ghcr.io/cirruslabs/macos-runner:sonoma)	5m 23s
Windows (free-threading) / build (arm64)	4m 7s
macOS / build and test (ghcr.io/cirruslabs/macos-runner:sonoma)	4m 1s
Windows / build and test (x64)	19m 36s
Windows / build and test (Win32)	17m 50s
Windows (free-threading) / build and test (x64)	25m 8s
macOS / build and test (macos-13)	12m 19s
Ubuntu / build and test (ubuntu-24.04-arm)	8m 18s
Ubuntu / build and test (ubuntu-24.04)	11m 8s
Ubuntu (free-threading) / build and test (ubuntu-24.04)	16m 6s
Ubuntu (free-threading) / build and test (ubuntu-24.04-arm)	10m 17s
Ubuntu (bolt) / build and test (ubuntu-24.04)	17m 52s
Thread sanitizer / Thread sanitizer	5m 42s
Thread sanitizer (free-threading) / Thread sanitizer	10m 17s
All required checks pass	1s
	3h 47m 24s

Provide feedback