Staff Application Security Engineer, Expert AI Reviewer (PhD)

About the role
Cobalt builds expert data and evaluation infrastructure for AI developers. We are recruiting senior application security engineers for a contract project that evaluates how well frontier AI models find and fix security flaws in real software. You will design realistic application security tasks in a command line environment that a leading AI model cannot solve, then review how the model fails.
What you will do
Design self-contained command line tasks drawn from real application security work, such as finding and patching injection flaws, fixing broken authentication and access control, auditing a codebase for unsafe deserialization or memory safety bugs, remediating vulnerable dependencies, and securing a build and deployment pipeline.
Build the task environment, including the vulnerable application or codebase, write a reference solution, and write automated tests that verify both that the flaw is fixed and that the application still works.
Run your task against a frontier AI model, review its attempt, and refine the task until the failure reflects a real gap in the model's capability rather than ambiguity or trick wording.
Work with reviewers to bring each task to acceptance.
Who we are looking for
A PhD in computer science, software security, or a closely related field.
At least 10 years of hands-on experience in application security, secure software engineering, or product security.
At least one publication, either academic (for example a peer-reviewed paper) or professional (for example a conference talk, a published vulnerability disclosure, or a widely used open-source tool).
Strong code review skills in at least two widely used languages, such as Python, JavaScript, Java, Go, C, or C++.
Fluency in the Linux command line, shell scripting, Docker, and Git.
The ability to write precise specifications that another engineer could follow without asking questions.
Why Cobalt AI:
Advance frontier AI where it counts. Apply your research expertise to the data that frontier labs cannot obtain any other way, where your reasoning directly shapes how the next generation of models works through technical problems.
Grow professionally. Expand your influence through evaluation projects, advisory roles, and research collaborations, while deepening your understanding of how frontier models are trained and assessed.
Work with a top-tier network. Collaborate with researchers from leading institutions and labs on high-impact, flexible work.
Set your own schedule. Flexible 10 to 40 hour weeks that fit around your research position and your life.
Competitive pay. Rates vary by project and are determined by a number of factors, including scope, skillset, and experience.

Work arrangement
No