Issue #409: Bug: "Open Source Contributions" resume section is silently dropped, but open_source is the highest-weighted scoring category (35/100)

interviewstreet/hiring-agent · commit e0ece80 · investigation verified · 2026-10-18 · investigation cost $0.05

Recommended: merge #410

feat: extract and score Open Source Contributions resume sections

Fixes the cause · judged from diff Merges clean · git check
5/5
citations verified in code
194 → 14
open PRs shortlisted by cited lines
1 fix
1 merge clean · 0 partly
$0.05
investigation cost

How it breaks

The path is the model's reasoning. Each box is a citation checked against the code.

1pdf.py:268 ✓
Extraction runs only six hardcoded sections
2pdf.py:270 ✓
Result dict has no open source key
3models.py:211 ✓
JSONResume schema lacks an OSS field
4transform.py:728 ✓
Text serializer outputs only schema fields
5projects.jinja:1 ✓
Only the projects prompt mentions OSS
where it shows

What's wrong

The pipeline has no open-source contributions section. Extraction only runs six hardcoded section prompts. The JSONResume schema has no field to hold such data. The text serializer only writes existing schema fields.
Show full explanationA dedicated "Open Source Contributions" section therefore never reaches the evaluator, and open_source is scored without it. Only the "projects" prompt (projects.jinja) was recently told to include open source work. That is a partial, indirect path, and it doesn't cover a separate section reliably. The report's line numbers are slightly off: pdf.py is at line 268, not 269, and JSONResume is at models.py:211, not 193. The mechanism matches the report.
Model confidence: high.

How to reproduce

  1. Create a PDF resume with a distinct 'Open Source Contributions' section listing merged PRs, such as litellm, llama_index and instructor.
  2. Run: python score.py ./resume/sample.pdf --role software_engineering_intern
  3. Open cache/resumecache_*.json and confirm no open-source contribution data is stored. Any such text appears at most as a project entry, and only if the projects prompt picked it up.
  4. Check the open_source category evidence. It says there are no direct contributions to external projects.
  5. Optionally, add a print of convert_json_resume_to_text(resume) and confirm no open source contributions section appears in the evaluator input.

Evidence

Each citation was checked against the code at e0ece80.

pdf.py2 citations
line 268
verified in code
sections = ["basics", "work", "education", "skills", "projects", "awards"]

Extraction only runs these six section prompts, so there is no open_source or contributions pass.

line 270
verified in code
complete_resume = {

The result dict has fixed keys, none for open source, so nothing else is collected.

models.py1 citation
line 211
verified in code
class JSONResume(BaseModel):

The schema has no open source contributions field (fields at lines 214-225), so the data has nowhere to be stored.

transform.py1 citation
line 728
verified in code
def convert_json_resume_to_text(resume_data: JSONResume) -> str:

The serializer only emits sections for existing schema fields (basics, work, education, skills, projects, ...), so the evaluator prompt cannot include open source contributions.

prompts/templates/projects.jinja1 citation
line 1
verified in code
This includes the projects they have built & the open source projects they have done and mentioned in the resume.

The only partial handling is inside the projects prompt. There is no dedicated section, so a separate OSS section is not reliably captured.

Pull requests

Verdicts are judged from the diff, not run. Merge status is a real git check against e0ece80.

194open PRs
14change a cited line or mention the issue
1change the cause
1fixes and merges clean
Merges clean
Conflicts or unknown
Fixes
—
Partly
—
—
Why each verdict
13 PRs that don't fix itShow
#41 · Fix issue #40 by Anubhav200311

It only changes how section text is segmented and passed to the six existing extractors, and adds no open-source section, schema field or serialization.

Shortlisted because: changes pdf.py:268, changes pdf.py:270, changes prompts/templates/projects.jinja:1

It adds certificates and volunteer_experience extractors, not an open source section, and it adds no schema or serializer support for either.

Shortlisted because: changes pdf.py:270

It adds certificates and publications section models and extractors, and nothing for open source.

Shortlisted because: changes pdf.py:270

It rewrites PDF text-extraction validation and reformats code, and the six-section list, schema and serializer stay as they were.

Shortlisted because: changes pdf.py:268, changes pdf.py:270

It adds a scholarships section through the same pipeline layers, but nothing for open source contributions, and projects.jinja is unchanged.

Shortlisted because: changes models.py:211, changes pdf.py:270

Its pdf.py and models.py changes are the same certificates and publications additions as #70, and it adds nothing for open source.

Shortlisted because: changes pdf.py:270

It replaces the per-section calls with a header-based splitter whose known_headers map has no open source entry, so an OSS section is not extracted. It also adds no schema field or serialization.

Shortlisted because: changes pdf.py:268, changes pdf.py:270

It fixes fork-contribution detection in the GitHub enrichment (github.py) and refactors LLM response parsing, so resume-section extraction, schema and serialization are untouched.

Shortlisted because: changes pdf.py:268

It adds a languages section, parallel extraction, and repo/live URL handling for projects, but no open source section or serialization.

Shortlisted because: changes pdf.py:270

It adds a publications extractor and CSV export, and the Publication model already exists in the schema, but nothing covers open source contributions.

Shortlisted because: changes pdf.py:268, changes pdf.py:270

It only makes sections other than basics optional on failure, and the section list still has no open source entry.

Shortlisted because: changes pdf.py:268, changes pdf.py:270

It adds a post-scoring resume rewriter and a scoring refactor, and the extraction sections, JSONResume schema and text serializer still lack open source.

Shortlisted because: changes models.py:211

It expands the prompt templates and only mentions Open Source as a silent project-type hint in projects.jinja, with no dedicated section, schema field or serializer change.

Shortlisted because: changes prompts/templates/projects.jinja:1

What this packet does not prove

  • Citations are checked against the code. The explanation that links them, the order of the chain, and any claim about outside systems (APIs, libraries) are the model's reasoning and are not checked.
  • PR verdicts are judged from the diff, not run. No PR was applied or tested.
  • PRs were shortlisted because they change a cited line or mention the issue. A fix in another file, or a PR whose line numbers have drifted, is missed, so this is not proof that no other PR fixes it.