Skip to content

Commit 4cea3eb

Browse files
committed
fix: verify OpenCV backends and isolate native wheel compilation
1 parent f25d7d1 commit 4cea3eb

4 files changed

Lines changed: 171 additions & 9 deletions

File tree

‎Dockerfile‎

Lines changed: 17 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -27,25 +27,33 @@ RUN apt-get update && apt-get upgrade -y \
2727
libgomp1 libgl1 libglib2.0-0t64 libxcb1 libfreetype6 libfontconfig1 \
2828
&& rm -rf /var/lib/apt/lists/*
2929

30-
FROM runtime-base AS ocr-build
30+
# Keep native compilation independent of the runtime security-refresh stage.
31+
FROM ${RUNTIME_IMAGE} AS opencv-build
3132
RUN apt-get update && apt-get install -y --no-install-recommends \
3233
build-essential python3-dev python3-venv \
3334
&& rm -rf /var/lib/apt/lists/*
34-
COPY ocr/requirements.txt /app/ocr/requirements.txt
35-
RUN --mount=type=cache,target=/root/.cache/pip python3 -m venv /opt/ocr \
36-
&& CMAKE_ARGS="-DWITH_FFMPEG=OFF -DWITH_GSTREAMER=OFF -DWITH_V4L=OFF -DWITH_QT=OFF -DWITH_GTK=OFF -DWITH_OPENGL=OFF -DBUILD_EXAMPLES=OFF" \
35+
RUN python3 -m venv /opt/opencv-build \
36+
&& CMAKE_ARGS="-DWITH_FFMPEG=OFF -DVIDEOIO_ENABLE_PLUGINS=OFF -DWITH_GSTREAMER=OFF -DWITH_V4L=OFF -DWITH_QT=OFF -DWITH_GTK=OFF -DWITH_OPENGL=OFF -DBUILD_EXAMPLES=OFF" \
3737
CMAKE_BUILD_PARALLEL_LEVEL=4 \
38-
/opt/ocr/bin/pip wheel --no-cache-dir --no-deps \
38+
/opt/opencv-build/bin/pip wheel --no-cache-dir --no-deps \
3939
--no-binary=opencv-python,opencv-python-headless \
40-
--wheel-dir=/tmp/opencv-wheelhouse \
41-
opencv-python==5.0.0.93 opencv-python-headless==5.0.0.93 \
40+
--wheel-dir=/opencv-wheelhouse \
41+
opencv-python==5.0.0.93 opencv-python-headless==5.0.0.93
42+
43+
FROM runtime-base AS ocr-build
44+
RUN apt-get update && apt-get install -y --no-install-recommends python3-venv \
45+
&& rm -rf /var/lib/apt/lists/*
46+
COPY --from=opencv-build /opencv-wheelhouse /tmp/opencv-wheelhouse
47+
COPY ocr/requirements.txt /app/ocr/requirements.txt
48+
RUN --mount=type=cache,target=/root/.cache/pip python3 -m venv /opt/ocr \
4249
&& /opt/ocr/bin/pip install --no-cache-dir --no-index --no-deps \
4350
--find-links=/tmp/opencv-wheelhouse \
4451
opencv-python==5.0.0.93 opencv-python-headless==5.0.0.93 \
4552
&& rm -rf /tmp/opencv-wheelhouse \
4653
&& /opt/ocr/bin/pip install -r /app/ocr/requirements.txt \
47-
&& /opt/ocr/bin/pip check \
48-
&& /opt/ocr/bin/python -c 'import cv2, re; info = cv2.getBuildInformation(); match = re.search(r"^\s*FFMPEG:\s*(\S+)", info, re.MULTILINE); assert match and match.group(1) == "NO", "OpenCV FFmpeg backend must be disabled"; print("OpenCV FFMPEG:", match.group(1))'
54+
&& /opt/ocr/bin/pip check
55+
COPY ocr/verify_opencv.py /app/ocr/verify_opencv.py
56+
RUN /opt/ocr/bin/python /app/ocr/verify_opencv.py
4957
COPY ocr/download_models.py ocr/models.json /app/ocr/
5058
RUN /opt/ocr/bin/python /app/ocr/download_models.py --directory /app/ocr/models --manifest /app/ocr/models.json
5159

‎docs/ci-recovery-2026-10-05.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,12 @@ Worker policy is resolved from the live native spawn catalog on October 5, 2026:
1818

1919
## Work and release ledger
2020

21+
The second implementation review accepted the actual verifier, its test/native-QA integration, the separated wheel builder, and compiler isolation. The reviewer confirmed that both strict workflow gates were unchanged and the diff passed whitespace checks. Primary reran the seven guard tests successfully after the final integration changes. Native image compatibility and complete-build security/storage qualification remain pending.
22+
23+
The correction separates the source-wheel builder from `runtime-base`, retaining the identical pinned Ubuntu/Python ABI while keeping compiler packages outside the runtime. FFmpeg and video-I/O plugins are disabled during compilation. `ocr/verify_opencv.py` then checks the actual imported wheel in a clean registry environment: FFmpeg must be absent from registered backends and unavailable, native FFmpeg libraries/plugins must be absent, and the imported extension's dependencies must resolve without FFmpeg. Affirmative or ambiguous build-information rows are rejected; an omitted row is insufficient without the registry and binary checks. Seven focused guard tests passed. Full local Python discovery could not import `pillow_heif`; native container qualification remains required. The immutable-image QA helper invokes that same packaged guard and includes its tests alongside the existing OCR tests.
24+
25+
The first corrected candidate was committed and pushed on `main` as `f25d7d15e22751a8f59eea14bbc1e8694e696acc`, triggering run `37310830635` on the intended 32 GiB lane. Both pinned OpenCV source wheels compiled successfully, and the locked dependency installation did not replace them. The build then failed at the installed OpenCV FFmpeg exclusion assertion; the scan and release-tag publication were skipped. Its 15,016,493,056-byte partial storage report is not a completed-image measurement and cannot qualify runner resizing. Backend verification is under correction, with no scan waiver or deployment.
26+
2127
The independent read-only candidate review completed with no static blocker in the Dockerfile, workflow, or dependency overrides. Its accepted result permits the qualification build on the existing 32 GiB lane. The review checked wheel build/install sequencing, the enforced `FFMPEG: NO` assertion, targeted base-stage cache invalidation, and preservation of both gates. It did not claim native runtime compatibility: the surviving `cv2` installation and all 11 PNG/JPEG/HEIC fixtures remain release prerequisites.
2228

2329
Recovery is in progress. The runner owner must confirm any newly requested storage label is configured and listening before a workflow push can target it. Vulnerabilities must be repaired against artifact evidence; scanner suppression and artificial disk padding are excluded. A green build must prove both gates and release-tag publication, then the resulting image must pass native OCR and runtime checks before the production handoff.

‎ocr/test_verify_opencv.py‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
from pathlib import Path
2+
import subprocess
3+
import sys
4+
import tempfile
5+
from types import SimpleNamespace
6+
import unittest
7+
from unittest.mock import patch
8+
9+
10+
sys.path.insert(0, str(Path(__file__).parent))
11+
import verify_opencv as verifier
12+
13+
14+
class OpenCvVerificationTest(unittest.TestCase):
15+
def setUp(self):
16+
self.directory = tempfile.TemporaryDirectory()
17+
self.addCleanup(self.directory.cleanup)
18+
self.prefix = Path(self.directory.name)
19+
self.package = self.prefix / "lib" / "cv2"
20+
self.package.mkdir(parents=True)
21+
(self.package / "__init__.py").touch()
22+
(self.package / "cv2.test.so").touch()
23+
self.cv2 = SimpleNamespace(__file__=str(self.package / "__init__.py"),
24+
__version__="5.0.0", CAP_FFMPEG=1900,
25+
getBuildInformation=lambda: "Video I/O:\n Images: YES\n",
26+
videoio_registry=SimpleNamespace(getBackends=lambda: (2000, 2200),
27+
hasBackend=lambda value: False))
28+
self.ldd = patch.object(verifier.subprocess, "run", return_value=
29+
subprocess.CompletedProcess([], 0, "libc.so.6 => /lib/libc.so.6", ""))
30+
self.ldd.start()
31+
self.addCleanup(self.ldd.stop)
32+
33+
def test_missing_text_still_requires_registry_and_native_inspection(self):
34+
report = verifier.verify(self.cv2, self.prefix)
35+
self.assertFalse(report["ffmpegAvailable"])
36+
self.assertEqual(report["nativeExtensions"], 1)
37+
self.assertEqual(report["ffmpegConfigurationValues"], [])
38+
verifier.subprocess.run.assert_called_once()
39+
40+
def test_disabled_configuration_spellings(self):
41+
for value in ("NO", "OFF", "false", "DISABLED"):
42+
with self.subTest(value=value):
43+
self.cv2.getBuildInformation = lambda: f" FFmpeg: {value}\n"
44+
verifier.verify(self.cv2, self.prefix)
45+
46+
def test_enabled_or_unknown_configuration_is_rejected(self):
47+
for value in ("YES", "ON", "true", "unknown"):
48+
with self.subTest(value=value):
49+
self.cv2.getBuildInformation = lambda: f" FFMPEG: {value}\n"
50+
with self.assertRaises(RuntimeError):
51+
verifier.verify(self.cv2, self.prefix)
52+
53+
def test_registered_but_unavailable_backend_is_rejected(self):
54+
self.cv2.videoio_registry.getBackends = lambda: (1900, 2000)
55+
with self.assertRaisesRegex(RuntimeError, "registers"):
56+
verifier.verify(self.cv2, self.prefix)
57+
58+
def test_available_backend_is_rejected(self):
59+
self.cv2.videoio_registry.hasBackend = lambda value: True
60+
with self.assertRaisesRegex(RuntimeError, "provides"):
61+
verifier.verify(self.cv2, self.prefix)
62+
63+
def test_ffmpeg_library_or_plugin_is_rejected(self):
64+
for filename in ("libavcodec-a123.so.62", "libopencv_videoio_ffmpeg.so"):
65+
with self.subTest(filename=filename):
66+
library = self.prefix / filename
67+
library.touch()
68+
with self.assertRaisesRegex(RuntimeError, "packaged"):
69+
verifier.verify(self.cv2, self.prefix)
70+
library.unlink()
71+
72+
def test_missing_or_ffmpeg_linked_dependencies_are_rejected(self):
73+
for output in ("libz.so.1 => not found", "libavutil.so.60 => /lib/libavutil.so.60"):
74+
with self.subTest(output=output):
75+
verifier.subprocess.run.return_value.stdout = output
76+
with self.assertRaisesRegex(RuntimeError, "dependencies"):
77+
verifier.verify(self.cv2, self.prefix)
78+
79+
80+
if __name__ == "__main__":
81+
unittest.main()

‎ocr/verify_opencv.py‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
"""Verify that the installed OpenCV cannot provide an FFmpeg backend."""
2+
3+
import json
4+
import os
5+
from pathlib import Path
6+
import re
7+
import subprocess
8+
import sys
9+
10+
11+
def verify(cv2, prefix):
12+
prefix = Path(prefix).resolve()
13+
module = Path(cv2.__file__).resolve()
14+
if not module.is_relative_to(prefix):
15+
raise RuntimeError("OpenCV was imported from outside the OCR environment")
16+
17+
info = cv2.getBuildInformation()
18+
values = re.findall(r"^\s*FFMPEG:\s*(\S+)", info, re.MULTILINE | re.IGNORECASE)
19+
if len(values) > 1 or any(value.upper() not in {"NO", "OFF", "FALSE", "DISABLED"}
20+
for value in values):
21+
raise RuntimeError(f"OpenCV reports enabled or ambiguous FFmpeg support: {values}")
22+
23+
backend = int(cv2.CAP_FFMPEG)
24+
registered = [int(value) for value in cv2.videoio_registry.getBackends()]
25+
available = bool(cv2.videoio_registry.hasBackend(backend))
26+
if backend in registered or available:
27+
raise RuntimeError(f"OpenCV registers or provides the FFmpeg backend: "
28+
f"registered={registered}, available={available}")
29+
30+
extensions = sorted(module.parent.rglob("cv2*.so"))
31+
if not extensions:
32+
raise RuntimeError("Installed OpenCV native extension was not found")
33+
forbidden = re.compile(r"ffmpeg|(?:^|[/\s])(?:lib)?(?:avcodec|avformat|avutil|avdevice|"
34+
r"avfilter|swscale|swresample|postproc)(?:[-_.\s]|$)", re.IGNORECASE)
35+
for library in prefix.rglob("*.so*"):
36+
if forbidden.search(library.name):
37+
raise RuntimeError(f"FFmpeg library or plugin is packaged: {library.name}")
38+
for extension in extensions:
39+
linked = subprocess.run(["ldd", str(extension)], capture_output=True, text=True,
40+
timeout=30, check=True)
41+
if "not found" in linked.stdout or forbidden.search(linked.stdout):
42+
raise RuntimeError("OpenCV native dependencies are missing or include FFmpeg: "
43+
+ linked.stdout.strip())
44+
45+
return {"version": cv2.__version__, "module": str(module), "ffmpegBackendId": backend,
46+
"registeredBackends": registered, "ffmpegAvailable": available,
47+
"ffmpegConfigurationValues": values, "nativeExtensions": len(extensions),
48+
"packagedFfmpegLibraries": 0, "linkedFfmpegLibraries": 0}
49+
50+
51+
def main():
52+
# Registry priorities can conceal a compiled backend. Inspect in a clean environment.
53+
for key in list(os.environ):
54+
if key.startswith("OPENCV_VIDEOIO_PRIORITY_") or key == "OPENCV_VIDEOIO_PLUGIN_PATH":
55+
del os.environ[key]
56+
import cv2
57+
58+
try:
59+
print(json.dumps(verify(cv2, sys.prefix)))
60+
except Exception as error:
61+
print(json.dumps({"error": str(error), "buildInformation": cv2.getBuildInformation()}),
62+
file=sys.stderr)
63+
raise
64+
65+
66+
if __name__ == "__main__":
67+
main()

0 commit comments

Comments
 (0)