Software Engineer – AI Training & Model Evaluation

Software Engineer – AI Training & Model EvaluationCompensation: $60–$120/hour equivalentLocation: Global | Fully RemoteEmployment Type: ContractTime Commitment: 15+ hours per weekSchedule: Flexible
About the RoleWe are seeking experienced Software Engineers with prior professional experience in AI training, model evaluation, or AI benchmarking to contribute to a project focused on improving advanced AI systems.In this role, you will leverage your software engineering expertise to design challenging programming tasks, evaluate AI-generated code, identify technical weaknesses, and establish objective standards for high-quality solutions.This opportunity is best suited for engineers who have previously worked on substantive AI-training projects and have a strong foundation in practical software development.
Key ResponsibilitiesDesign realistic and challenging software engineering tasks to evaluate advanced AI models.Develop detailed task instructions, reproducible environments, reference solutions, and automated tests.Review and evaluate AI-generated code for correctness, robustness, efficiency, and completeness.Identify technical issues related to implementation, debugging, dependencies, performance, and reproducibility.Create objective evaluation criteria and grading standards for technical submissions.Assess model performance across different programming and software engineering scenarios.
Required QualificationsPrior paid professional experience in AI training, model evaluation, benchmarking, or improving AI systems.Strong hands-on proficiency in Python.Professional software engineering experience with at least one additional programming language, such as Java, JavaScript, TypeScript, Rust, C++, C#, or Go.Strong understanding of software design, debugging, testing, version control, and dependency management.Ability to assess complex codebases and identify technical issues independently.
Public Proof of Technical SkillsApplicants should be able to demonstrate their technical expertise through publicly accessible evidence, such as:Substantial GitHub or GitLab repositories.Meaningful open-source contributions.Maintainer or core-contributor experience.Competitive programming achievements (Codeforces, ICPC, Olympiads, etc.).Publicly developed tools, libraries, or software systems.Technical publications, benchmark contributions, or engineering write-ups.
Preferred QualificationsExperience creating AI-training tasks, reference solutions, automated tests, graders, or benchmark problems.Experience reviewing and evaluating technical submissions from other AI trainers.Codeforces rating of approximately 1900+ or equivalent competitive programming achievements.Familiarity with Linux, Docker, CI/CD, command-line tools, and sandboxed development environments.Experience with long-horizon or agentic coding evaluations.Availability to start immediately.

Skills
Python, JavaScript, TypeScript, Java, C++, C#, Rust

Work arrangement
Yes