Live opening · Posted 3 days ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
Position: Member of Technical Staff, Coding Research
Type: Full-time
Compensation: $220K - $500K/yr
Location: Remote
Role Responsibilities
Design and own evaluation frameworks for coding agents, including benchmark specifications, scoring methodologies, rubrics, and quality standards.
Lead end-to-end research initiatives focused on measuring and improving coding model performance across diverse software engineering tasks.
Develop high-quality datasets, golden examples, and evaluation protocols that enable reliable assessment of frontier coding systems.
Analyze model behavior and failure modes, identifying systematic weaknesses and translating findings into actionable improvements for training and evaluation.
Build tooling and infrastructure that support large-scale experimentation, data generation, review workflows, and evaluation pipelines.
Establish best practices for coding-agent assessment, ensuring methodological rigor, reproducibility, and measurement quality.
Requirements
Have a strong software engineering background with expertise in Python, C++, or comparable programming languages.
Have strong relevant experience in software engineering, machine learning, AI research, evaluation, or related technical disciplines.
Have experience designing, reviewing, or validating technical assessments, benchmarks, coding tasks, or evaluation methodologies.
Have familiarity with large language models, coding agents, reinforcement learning, model evaluation, or related AI systems.
Have a proven ability to build tooling, automate workflows, and improve technical processes through systematic experimentation.
Application Process
Easy Apply on LinkedIn
Check email for next steps
Participate in resume evaluation & interview stage
Work arrangement
Yes
Recently tracked roles with full details and direct application links.