Live opening · Posted 2 days ago

AI Evaluation and Automation Engineer

Tranzeal Incorporated · United States (Remote)

Linkedin Yes

JobBeeper subscribers received an alert for this role.

At a glance

The key details from the original listing.

Posted 2 days ago

CompanyTranzeal Incorporated

LocationUnited States (Remote)

Work modeYes

SkillsPython, JavaScript, TypeScript, Docker

Source

ListedPosted 2 days ago

Your early-applicant advantage

Live timing from JobBeeper.

Live data

6 min from Linkedin publishing this role to us finding it

13 min median time from a role going live to a subscriber being told

6 hours subscribers had this role before this page existed

63,132 roles found in the last 24 hours — the newest are not on this site yet

Start your free trial →

About the role

Description supplied by the original job listing.

Role: AI Evaluation & Automation Engineer
Location: San Diego, CA
Work Mode: Remote
Schedule: PST Hours
We’re seeking a Software Engineer to build and automate evaluation systems for AI-powered software development tools.
MUST-HAVES:
Python or JavaScript/TypeScript
AI Evaluation Automation / Benchmarking / Evaluation Harnesses
Hands-on experience with AI coding/agentic tools — Claude Code, Devin, Cursor, OpenCode, or similar
Strong Git knowledge, including branches, history & working trees
Hands-on Docker / Containerization
Experience with APIs, CI/CD, automation, testing & validation
Understanding of LLM/AI/agent evaluation, reproducibility, and evaluation validity
Nice to Have: Execution-based grading, LLM-as-Judge/Agent-as-Judge, SWE-bench, Bazel, evaluation datasets.
If you have hands-on AI coding-agent + AI evaluation automation experience, we’d like to hear from you.

Work arrangement
Yes

Get JobBeeper Mobile App

Never miss a job opening! Get instant job alerts on your phone.

Subscribers see fresh openings within minutes. Download the JobBeeper App on Google Play to get real-time push notifications and apply before anyone else.

⚡ Instant Push Alerts 🎯 Tailored Filters 🚀 Direct Employer Links

GET IT ON Google Play

More openings worth a look

Recently tracked roles with full details and direct application links.

6 roles

ACCOUNT MANAGER – US IT RECRUITMENT Tranzeal Incorporated · Hyderabad, Telangana, India (On-site) Resource Management Partner – Budget & FP&A Operations (with exp in Allocadia / Uptempo platform) Tranzeal Incorporated · United States (Remote) Principal SQL Server Performance Engineer Tranzeal Incorporated · Miami, FL (Remote) Resource Management Partner – Budget & FP&A Operations (with exp in Allocadia / Uptempo platform) Tranzeal Incorporated · United States (Remote) Senior Palantir Foundry Developer | Remote | W2 Contract Tranzeal Incorporated · United States (Remote) Alliance Partnership Analyst Tranzeal Incorporated · Los Angeles, CA (Remote)

Good roles move before most people even see them. Tell JobBeeper what you want and get fresh matches delivered in minutes.

Start your free trial →