Skip to content

[Parser] Double periods from OCR not cleaned up #14

Description

@ilhamfp

Parser Improvement: Double periods from OCR not cleaned up

Severity: LOW
File: ocr_correct.py
Function: correct_ocr_errors
Estimated errors fixed: 1

Current Behavior

List items sometimes get double periods (e.g., 'i..') due to OCR artifacts or tokenization issues at sentence boundaries. These are not cleaned up by any existing pattern.

Proposed Fix

Add a pattern to normalize double (or more) periods that aren't part of an ellipsis (three dots) into single periods. Be careful not to affect legitimate ellipsis ('...').

Code Before

    # Common scanner artifacts
    (re.compile(r'^[;,.]$', re.MULTILINE), ''),  # Lone punctuation on a line

Code After

    # Double period fix (but preserve ellipsis '...')
    (re.compile(r'(?<![.])\.{2}(?![.])'), '.'),  # Exactly two dots -> one dot

    # Common scanner artifacts
    (re.compile(r'^[;,.]$', re.MULTILINE), ''),  # Lone punctuation on a line

Generated by the Pasal.id Correction Agent (Opus 4.6) after analyzing 4 parser feedback entries.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions