Repository navigation
feat(backend): 视频理解支持场景检测智能抽帧 - #465
Open
TomRush-qn wants to merge 1 commit into
Open
TomRush-qn wants to merge 1 commit into
TomRush-qn wants to merge 1 commit into
Conversation
抽帧计划优先使用 ffmpeg 场景切换检测(select='gt(scene,t)')按镜头 切点采样,替代固定间隔盲采: - 始终包含首帧;相邻采样点带最小间隔(frame_interval/2)防抖; 保留 max_frames 上限 - 检测失败或未检出切换(纯静止视频)时回退原固定间隔逻辑,行为 完全兼容;smart_sampling=False 可显式关闭 - 新增单测覆盖 pts_time 解析、采样计划边界、场景采样/无切点回退/ 关闭开关三条路径,并通过真实 ffmpeg 对 24s 合成视频验证切点检出 Closes JefferyHcool#464
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #464
改动内容
backend/app/utils/video_reader.py(约 60 行)+ 新增单测,不触碰拼图、导出等下游逻辑:_detect_scene_cuts():用 ffmpegselect='gt(scene,threshold)'+metadata=print:file=-检测场景切点,把pts_time解析为整秒时间戳;ffmpeg 失败/超时返回空列表_plan_scene_timestamps():切点 → 抽帧计划,始终包含首帧、相邻采样点最小间隔max(1, frame_interval // 2)防抖、保留max_frames上限、丢弃超出时长的切点extract_frames():smart_sampling=True(默认)且检出切点时按切点抽帧;未检出切换(纯静止视频)或检测失败时回退原固定间隔逻辑smart_sampling/scene_threshold,默认行为变化仅作用于视频理解抽帧路径,smart_sampling=False可完全还原旧行为为什么默认开启
固定间隔 + 相邻 MD5 精确去重只对"整帧逐字节相同"的全静止画面有效;最常见的"PPT + 讲师头像"课程形态下人物微动即导致去重失效,一小时 5s 采样会产生 720 帧全部送入多模态模型。切点采样以更少帧数覆盖更多有效信息,显著降低图片 token 成本。回退逻辑保证纯静止视频与 ffmpeg 异常时的兼容性。如维护者倾向保守,可将默认值改为
False并在后续版本由前端暴露开关——改动仅一处构造参数。测试
backend/tests/test_video_reader_smart_sampling.py(8 个用例,沿用test_video_reader_dedupe.py的 stub 模式):pts_time解析、ffmpeg 失败返回空、采样计划(首帧/最小间隔/max_frames 上限/越界切点)、按切点抽帧、无切点回退固定间隔、关闭开关时检测不执行test_video_reader_dedupe.py通过[8, 16]✅python -m unittest tests.test_video_reader_smart_sampling tests.test_video_reader_dedupe→ OK (9 tests)