We are looking for software engineers to validate real-world repositories used to evaluate large language models. You will triage issues, set up reproducible environments, assess test quality, and check how well models fix real bugs.
About the role
We are looking for software engineers to validate real-world repositories used to evaluate large language models. You will triage issues, set up reproducible environments, assess test quality, and check how well models fix real bugs.
Key Responsibilities
Review and triage issues in open-source GitHub projects.
Prepare repositories for evaluation, including containerisation and environment automation.
Assess how thorough and well written the unit tests are.
Edit and run code locally to see how LLMs perform on bug-fixing tasks.
Help pick repositories and issues that are hard for LLMs.
Write up findings clearly for researchers and reviewers.
Follow project guidelines, take part in calibration and review sessions, and incorporate feedback to keep quality consistent.
Official source: www.rex.zone. Details can change; the platform's page is the final word. How we check listings.
