Skip to content

补回 2/28~3/2 丢失的论文数据 #3

Description

@CMander02

背景

3月3日(Tue)的 GitHub Actions workflow 因超时(2h)被取消,导致 2/28、3/1、3/2 三天的论文未被抓取。

丢失数据量(dry-run 统计)

日期 待处理论文 预估耗时
2026-02-28 43 篇 ~45 min
2026-03-01 70 篇 ~70 min
2026-03-02 103 篇 ~105 min
合计 216 篇 ~3.5-4 小时

补回方案

分三次本地执行,避免超时:

uv run python scripts/fetch_papers.py --date 2026-02-28
uv run python scripts/fetch_papers.py --date 2026-03-01
uv run python scripts/fetch_papers.py --date 2026-03-02

根因

3/2 周一 arxiv 发布包含周末积压论文,3/3 Tue workflow 需处理 3 天(2/28~3/2)的论文量(216 篇),超出 2h timeout。

预防

考虑将 workflow timeout 从 180min 提高,或在周二分批处理。


Labels: bug, data

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions