Live opening · Posted 6 hours ago
The key details from the original listing.
Live timing from JobBeeper.
Description supplied by the original job listing.
Hirely is recruiting Software Engineers on behalf of a hiring partner for a project focused on evaluating advanced AI system outputs.
You'll use your software-engineering expertise to assess AI-generated technical content, including documentation, code comments, error messages, architectural explanations, and technical instructions. The focus is on identifying subtle technical errors, communication issues, ambiguities, and misleading answers.
No prior AI experience is required.
What You'll Do
Evaluate and compare AI-generated responses to technical prompts
Assess responses for technical correctness, clarity, tone, and usefulness
Identify technically accurate responses that are poorly communicated or difficult to understand
Detect fluent but factually incorrect, incomplete, or misleading technical responses
Assign quality scores based on defined evaluation criteria
Provide detailed written rationales explaining your assessments
Identify subtle errors, inconsistencies, ambiguities, and misleading technical statements
Review technical documentation, API references, design explanations, code comments, and instructions
Document findings with accuracy, consistency, and attention to detail
Collaborate asynchronously with project managers and reviewers to maintain evaluation standards
Work independently to meet project requirements and deliverables
Who We're Looking For
Native-level or excellent English proficiency with strong written communication skills
Professional experience as a Software Engineer or in a closely related technical role
Experience with modern software-development technologies and practices
Experience writing or reviewing technical documentation, design documents, API references, or specifications
Experience conducting code reviews, design reviews, or technical feedback
Strong understanding of software engineering concepts and terminology
Ability to evaluate technical communication for accuracy and audience appropriateness
Strong analytical and critical-thinking skills
Exceptional attention to detail
Ability to work independently in a remote environment
Experience with AI model evaluation, annotation, or RLHF projects is a plus but not required
Compensation & Terms
$90-140/hour, independent contractor
Fully remote
Flexible schedule
No prior AI experience required
How To Apply
Apply through the link. The process may include screening questions, an AI interview, a technical assessment, and hiring manager review.
About Hirely
Hirely connects professionals with remote opportunities from hiring partners across artificial intelligence, software engineering, cloud, data, research and other specialised fields. Hirely acts as a recruitment and talent-sourcing partner for the hiring companies it represents. Final hiring decisions, onboarding, compensation and project assignments are determined by the hiring partner.
Work arrangement
No
Recently tracked roles with full details and direct application links.