Principal Security Researcher, AI Model Evaluation (PhD, Contract)

About the role
Cobalt builds expert data and evaluation infrastructure for AI developers. We are recruiting senior security researchers for a contract project that evaluates how well frontier AI models perform real technical work in a command line environment. Your job is to design original, realistic security tasks that a leading AI model fails to solve. Accepted tasks are used to measure and improve the capabilities of frontier models.
What you will do
Design self-contained command line tasks drawn from real security work, such as vulnerability analysis, exploit development, reverse engineering, cryptographic implementation review, incident forensics, and system hardening.
Build the task environment, write a reference solution, and write automated tests that verify whether a solution is correct.
Test your task against a frontier AI model and refine it until the model fails for substantive reasons rather than because of ambiguity or trick wording.
Respond to reviewer feedback until the task is accepted.
Who we are looking for
A PhD in computer science, security, or a closely related field.
Industry or academic experience in security research or security engineering.
At least one publication, either academic (for example a peer-reviewed paper) or professional (for example a conference talk, a published vulnerability disclosure, or a widely used open-source tool).
Fluency in the Linux command line, shell scripting, Docker, and Python.
The ability to write precise task specifications that another expert could follow without asking questions.
Why Cobalt AI:
Advance frontier AI where it counts. Apply your research expertise to the data that frontier labs cannot obtain any other way, where your reasoning directly shapes how the next generation of models works through technical problems.
Grow professionally. Expand your influence through evaluation projects, advisory roles, and research collaborations, while deepening your understanding of how frontier models are trained and assessed.
Work with a top-tier network. Collaborate with researchers from leading institutions and labs on high-impact, flexible work.
Set your own schedule. Flexible 10 to 40 hour weeks that fit around your research position and your life.
Competitive pay. Rates vary by project and are determined by a number of factors, including scope, skillset, and experience.

Work arrangement
No