Fix Garbage Characters in Resume Parser

[pisum-sativum]

Description

Fixes #6096

This PR resolves an issue with the resume parsing engine where valid UTF-8 characters were aggressively stripped out while misencoded garbage characters and decoding artifacts (such as é, \uFFFD) polluted the extracted text.

Key changes include:

• Replaced the overly restrictive ASCII-only regex with one that safely preserves valid UTF-8, formatting marks, and bullet points.
• Fixed a bug where failed parsing of binary documents (PDF/DOCX) improperly fell back to decoding raw bytes as UTF-8, causing massive artifact output.
• Introduced a heuristic that cleanly repairs common misencoded UTF-8 artifacts resulting from document processor font-lookup issues without disrupting valid characters.

Pillar

• 🎨 Pillar 1 — New Theme Design
• 📐 Pillar 2 — Geometric SVG Improvement
• 🕐 Pillar 3 — Timezone Logic Optimization
• 🛠️ Other (Bug fix, refactoring, docs)

Visual Preview

(Not applicable as this is a backend parser fix, but the dashboard preview will no longer contain garbage formatting artifacts.)

Checklist before requesting a review:

• I have read the CONTRIBUTING.md file.
• I have tested these changes locally (localhost:3000/api/streak?user=YOUR_USERNAME).
• I have run npm run format and npm run lint locally and resolved all errors (CI will fail otherwise).
• My commits follow the Conventional Commits format (e.g., feat(themes): ..., fix(calculate): ...).
• I have updated README.md if I added a new theme or URL parameter.
• I have started the repo.
• I have made sure that i have only one commit to merge in this PR.
• The SVG output matches the CommitPulse "premium quality" aesthetic standard (no raw elements, smooth animations, correct fonts).
• (Recommended) I joined the CommitPulse Discord community for contributor discussions, mentorship, and faster PR support.

[vercel]

@pisum-sativum is attempting to deploy a commit to the jhasourav07's projects Team on Vercel.

A member of the Team first needs to authorize it.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[retenta-bot]

This pull request resolves a critical issue with the resume parser by enhancing its ability to handle UTF-8 characters and removing unwanted artifacts. This aligns with our previous decisions on bug fixes aimed at improving code quality, such as the recent fix for dead code in the calculateStreak function. Thank you for your contribution, and I look forward to your updates!

[pisum-sativum]

@JhaSourav07 I have done the necessary changes kindly check and merge with appropriate labels.

[github-actions]

📦 Next.js Bundle Size Report (Gzipped Sizes)

✨ No significant bundle size changes detected.

📊 Summary of Totals

Category	PR Size	Base Size	Difference
Total JS	3984.87 KB	3984.83 KB	+0.04 KB (+0.00%)
Total CSS	331.41 KB	331.41 KB	0 B

[Aamod007]

Thanks for fixing the resume parser encoding issue! Handling Latin-1 character mangling is a tricky bug, and the fallback parsing logic looks solid.

However, it looks like this PR accidentally includes some unrelated changes. You have modified files like \lib/github.ts, \services/github/ci-analytics.ts, and \�pp/generator/data/socials.ts, which don't seem related to the resume parsing fix.

Please revert the changes to those files so that the PR focuses solely on \lib/resume-parser.ts.

Looking forward to getting this merged once the branch is cleaned up!

Label Justification:

• \level:intermediate: Debugging and writing fixes for encoding issues and buffer parsing is an intermediate task.
• \ ype:bug: Fixes broken character rendering when parsing CV files.
• \mentor:Aamod007: Assigned as required.

[github-actions]

🚨 Hey @pisum-sativum, the CI Pipeline is failing on this PR and it has been marked as status:blocked.

Please fix the issues before this can be reviewed. Here's how:

1. Run checks locally before pushing:

npm run format:check   # Check Prettier formatting
npm run lint           # Run ESLint
npm run typecheck      # TypeScript type check
npm run test           # Run unit tests (Vitest)
npm run build          # Verify production build passes

2. Auto-fix common issues:

npm run format         # Auto-fix formatting with Prettier
npm run lint -- --fix  # Auto-fix lint errors where possible

3. Check the full failure log here:
👉 View CI Run

Once you push a fix and the CI passes, the status:blocked label will be removed automatically. 💪

[pisum-sativum]

Thanks for fixing the resume parser encoding issue! Handling Latin-1 character mangling is a tricky bug, and the fallback parsing logic looks solid.

However, it looks like this PR accidentally includes some unrelated changes. You have modified files like \lib/github.ts, \services/github/ci-analytics.ts, and \�pp/generator/data/socials.ts, which don't seem related to the resume parsing fix.

Please revert the changes to those files so that the PR focuses solely on \lib/resume-parser.ts.

Looking forward to getting this merged once the branch is cleaned up!

Label Justification:

• \level:intermediate: Debugging and writing fixes for encoding issues and buffer parsing is an intermediate task.
• \ ype:bug: Fixes broken character rendering when parsing CV files.
• \mentor:Aamod007: Assigned as required.

@Aamod007 I have done the necessary changes required but the CI is failing again and again I tried to fix it but I don't think so I can. Maybe after sometime will try ;(

[Aamod007]

I see you've pushed new commits since my last review. The resume parser changes look good — the latin1 re-encoding fix (Buffer.from(rawText, 'latin1').toString('utf-8')) is a clean approach for handling garbage characters, and restructuring the PDF/DOCX error handling to avoid the double-UTF8 fallback bug is a solid improvement.

However, CI checks are still failing (Format · Lint · Typecheck · Test). Please take a look at the failure logs and push a fix so we can move forward.

Label justification:

• level:intermediate — spans 4 files with non-trivial parser logic changes and formatting fixes
• quality:clean — clean code structure
• type:bug — fixes actual rendering bugs in resume parsing
• mentor:Aamod007 — assigned

[Aamod007]

Re-reviewing: the only CI failure was Vercel authorization (not code-related). All code checks pass. The Vercel deploy auth is a repo-level config issue. Approving.

[pisum-sativum]

@Aamod007 @JhaSourav07 Kindly check and merge if possible with correct labels.

[pisum-sativum]

@Aamod007 @JhaSourav07 Kindly merge this pr so that i can take up new issues since the bots is not allowing me to take up any more new issues .

[pisum-sativum]

@Aamod007 @JhaSourav07 If no change is required then kindly check and merge.

[pisum-sativum]

@Aamod007 @JhaSourav07 pls check and merge

[github-actions]

⚠️ Hey @pisum-sativum, this pull request has been inactive for 3 days. It will be automatically closed in 2 days if no further activity occurs.

If you are still working on this, please push your latest changes or leave a comment to keep it active.

[Aamod007]

Good catch on the parser fallback logic and UTF-8 handling! Ensuring that resume text is clean from encoding artifacts makes the dashboard experience much better for users. Note that there are some unrelated Prettier formatting changes in other files, but since they improve code formatting, it's fine. Approved!

[pisum-sativum]

@Aamod007 @JhaSourav07 Pls merge this pr asap

[pisum-sativum]

@Aamod007 @JhaSourav07 pls merge this pr asap

[pisum-sativum]

@JhaSourav07 could you please merge all the PRs I've worked on? I've been requesting this for several days now but they are still pending. I would really appreciate it if you could review and merge them as soon as possible. Thank you!

[github-actions]

⚠️ Hey @pisum-sativum, this pull request has been inactive for 3 days. It will be automatically closed in 2 days if no further activity occurs.

If you are still working on this, please push your latest changes or leave a comment to keep it active.

[pisum-sativum]

@JhaSourav07 pls merge

[pisum-sativum]

@JhaSourav07 Any update?

[github-actions]

⚠️ Hey @pisum-sativum, this pull request has been inactive for 3 days. It will be automatically closed in 2 days if no further activity occurs.

If you are still working on this, please push your latest changes or leave a comment to keep it active.

[github-actions]

⚠️ Hey @pisum-sativum, this pull request has been inactive for 3 days. It will be automatically closed in 2 days if no further activity occurs.

If you are still working on this, please push your latest changes or leave a comment to keep it active.

[github-actions]

⚠️ Hey @pisum-sativum, this pull request has been inactive for 3 days. It will be automatically closed in 2 days if no further activity occurs.

If you are still working on this, please push your latest changes or leave a comment to keep it active.

[github-actions]

⚠️ Hey @pisum-sativum, this pull request has been inactive for 4 days. It will be automatically closed in 1 days if no further activity occurs.

If you are still working on this, please push your latest changes or leave a comment to keep it active.