Issue #131: Issue: Inaccurate Open Source Contribution Detection
No open PR fixes this and merges clean
2 fix it but don't merge clean, 7 fix it partly. See the board below.
How it breaks
The path is the model's reasoning. Each box is a citation checked against the code.
What's wrong
github.py only lists the user's own repos via /users/{username}/repos and classifies each as open_source solely when the repo has more than one contributor. It never queries pull requests (merged or otherwise) to repos the user doesn't own.Show full explanation
So external merged PRs, PR merge status and star-based impact are never collected. The report's claims 1, 3 and 4 refer to PR-search code that isn't in this checkout, so I can't confirm them directly. Their cause is that no PR analysis exists at all. The undefined `response` bug (claim 5) is not present: the 404 branches use `status_code` from `_fetch_github_api`.How to reproduce
- Pick a GitHub user with merged PRs to repos they don't own and few or no own multi-contributor repos.
- Run the hiring-agent pipeline on a resume that links that GitHub profile.
- Inspect the output of fetch_all_github_repos. It contains only the user's own repos. No external PRs or merged-PR counts appear.
- A solo-maintained but popular repo (many stars/forks) is labelled project_type 'self_project' because contributor_count is 1.
Evidence
Each citation was checked against the code at e0ece80.
The only GitHub data source is the user's own repo list. No PR or search endpoint is used, so external contributions are never fetched.
Open-source classification begins here and depends on the contributor count alone.
Stars, forks, topics and fork work are ignored, so single-contributor open source projects are labelled self_project.
The 404 branch uses the defined status_code, so the reported undefined-variable bug is not present in this code.
The same defined variable is used in the repos path, so the reported bug is not present here either.
Pull requests
Verdicts are judged from the diff, not run. Merge status is a real git check against e0ece80.
fetch_user_pull_requests finds PRs to non-owned repos, reads real merged status from the pulls endpoint, collects repo stars/forks/license, and determine_project_type replaces the contributor-count rule.
Adds search-API fetches for merged, open and closed-unmerged PRs, splits external from own-repo PRs, pulls each target repo's stars/forks, and replaces the contributor-count-only classifier.
It collects merged PRs and upstream repo stars, but only for repos the user forked; PRs from non-forked workflows are still missed and non-fork classification is still contributor_count > 1.
It uses the parent repo's stars/forks for forked repos, which gives some star-based impact, but it adds no PR or merge-status collection and keeps the contributor-count classification.
It substitutes upstream parent stars/forks/topics for forks, but it still has no PR search or merge status and keeps contributor_count > 1 for classification.
It takes stars and forks from the parent repo for forks, but it adds no PR or merge detection and keeps the contributor-count rule.
For forked repos it searches merged PRs against the upstream repo and marks them open_source, but it covers only fork-based contributions, not PRs made without a fork, and non-fork classification is unchanged.
It takes stars and forks from the parent repo for forks, but it adds no PR or merge detection and still skips forks whose parent has fewer than 5 forks.
fetch_upstream_prs searches PRs to repos the user doesn't own and records merged state, but it doesn't collect star/impact data, is capped at 30, and leaves the contributor-count classification untouched.
8 PRs that don't fix itShow
It adds retry and rate-limit handling and fixes the status_code references, none of which touches PR collection or open-source classification.
It only adds retry, rate-limit and logging changes to _fetch_github_api and fixes the error branches, with no PR collection or classification change.
It changes cache file encoding and error handling and swaps the response references for status_code, with no PR or classification logic.
It fixes dead code in fetch_repo_contributors and improves logging, leaving the contributor-count classification and the absence of PR queries untouched.
It reformats console output and fixes the response variable, with no change to what data is collected.
It adds None handling for LLM JSON parsing and the status_code fix, which has nothing to do with PR collection or classification.
It only swaps response for status_code and fixes the contributors return type, neither of which collects PRs or changes classification.
It adds commit-timing forensics per owned repo and keeps the contributor-count classification, with no PR or external-contribution collection.
What this packet does not prove
- Citations are checked against the code. The explanation that links them, the order of the chain, and any claim about outside systems (APIs, libraries) are the model's reasoning and are not checked.
- PR verdicts are judged from the diff, not run. No PR was applied or tested.
- PRs were shortlisted because they change a cited line or mention the issue. A fix in another file, or a PR whose line numbers have drifted, is missed, so this is not proof that no other PR fixes it.