ML Engineer: Inference Optimization and GPU Kernels

Company Description: Anthology is a community of exceptional engineers and scientists ("Fellows") focused on training the next generation of AI models. Fellows use domain expertise to identify where current models fail, and then create high-quality training environments and data to teach AI how to improve on realistic engineering and research workflows.
We work with leading labs to accelerate progress on the hardest problems in software engineering, biology, medicine, materials science, energy, and beyond. Anthology is part of Polymath, which is backed by Base10, Founders Future, Y Combinator, and other incredible investors.
Role Description: Anthology is looking for experienced GPU kernel and model inference engineers to create and peer-review technical tasks for training and evaluating AI coding agents. These tasks focus on the work that makes models faster and cheaper to run: writing and tuning GPU kernels, and optimizing how models are served.
As an Anthology Fellow, you'll turn problems from your area of expertise into tasks with clear objectives, working reference solutions, and verifiers. You'll also review tasks created by other Fellows, checking their technical correctness, difficulty, and quality.
You should have experience in one ore more of the following:
GPU kernels: CUDA, Triton, CUTLASS, or similar; tiling, memory hierarchy, tensor cores, and operator fusion
Performance analysis: profiling with Nsight or similar tools, roofline analysis, and finding memory or compute bottlenecks
Inference systems: vLLM, SGLang, TensorRT-LLM, or similar; batching, KV cache management, paged attention, and scheduling
Model compression: quantization (INT8, FP8, FP4), sparsity, and their effect on accuracy
Decoding: speculative decoding, parallel decoding, and other ways to cut latency
In this role, you will:
Propose original tasks grounded in realistic kernel and inference optimization problems.
Write specifications that define the target workload, hardware, constraints, success criteria, and intended failure modes.
Build reproducible environments, starting code, baseline implementations, and reference solutions.
Develop automated evaluators that check numerical correctness and measure performance, such as speedup, throughput, latency, and memory use.
Test tasks against AI agent trials and investigate unexpected results.
Peer-review other Fellows' proposals, implementations, and evaluators within your expertise.
Run reference solutions, check edge cases, and identify ambiguous requirements, unreliable measurements, or shortcuts that could pass evaluation without solving the intended problem.
Give specific, actionable feedback and revise your own tasks in response to review.
You should bring:
Substantial engineering, research, or open-source experience writing or optimizing GPU kernels or inference systems.
A record of measurable speedups on real workloads, and the ability to explain where they came from.
Strong testing skills, including checking numerical correctness across data types, shapes, and edge cases.
Clear technical writing and comfort with reviewing code, questioning assumptions, and discussing technical trade-offs.
Contributions to projects like vLLM, SGLang, FlashAttention, Triton, CUTLASS, or PyTorch are useful, as is experience building benchmarks or reinforcement learning environments.
Compensation:
U.S. / Canada / UK / Western Europe / Australia / Singapore / Israel: $50–$150/hr
Compensation is based on experience, demonstrated technical depth, domain specialization, location, and the complexity and quality of work you are able to independently own.
How to apply: Submit an application on our website here!
https://anthologyresearch.ai/roles/ml-systems-engineer-kernels-inference-1a4903
After that, the first step is a casual AI voice interview, around 20 minutes long. We’ll discuss your background and go through some technical questions related to your field. No preparation is required to succeed in the interview.

Work arrangement
Yes