Public listing
Machine Learning & CPU Performance Engineer (Senior)
- Experience5+ yrs
- Work modeYes
- ListedPosted 8 hours ago
- SourceLinkedin
Role overview
About the role
Description supplied by the original job listing.
ABOUT US:
Differential AI (DiffAI) is an Ivy League PhD-inspired AI/ML startup that focuses on efficient, secure knowledge retrieval using smaller, highly optimized models that significantly reduce training and inference costs. Through our proprietary training and inference techniques, the platform offers high-accuracy, edge-ready AI solutions that can be deployed entirely on-premise or in single-tenant environments. Founded on principles derived from brand new PhD research, our technology offers compelling value propositions:
Private & Secure: Models run entirely offline, ensuring complete data privacy
Ultra-Efficient: 7,000x smaller models than conventional solutions
Cost-Effective: Up to 83% reduction in training costs
We are a post-revenue company with 2 patents pending. We are partnered with three F500 companies to deliver AI research projects. We are purposely bootstrapped after turning down a few investor indications. Our founding team consists of 2 AI/ML/Data Science PhDs and 1 MBA, with advanced degrees from Brown, Harvard, and Georgetown. We operate fully remotely.
YOUR RESPONSIBILITIES:
We are seeking someone to join our team in conducting systematic research on machine learning execution across CPU architectures. In this role, you will lead the implementation, runtime optimization, and micro-profiling of diverse ML model architectures purely on CPU backends (x86, ARM, multi-core server processors). You will build automated benchmarking tools to measure trade-offs between accuracy, latency, thread efficiency, memory bandwidth, and cache behavior.
CPU-Specific Runtime Optimization: Compile and run deep learning architectures using CPU-optimized runtimes (OpenVINO, ONNX Runtime, PyTorch CPU Inductor, llama.cpp / ggml, C++ implementations).
Thread & Memory Control: Configure and profile NUMA node topology, CPU core affinity, thread pinning, and thread pools (OpenMP, TBB) to isolate system variance and maximize throughput.
Instruction-Level Benchmarking: Profile and exploit low-precision CPU instructions (AVX-512, Intel AMX, ARM NEON/SVE) to evaluate low-bit quantization (INT8, FP8, BF16) versus floating-point precision on accuracy and speed.
Hardware Telemetry & Profiling: Conduct deep bottleneck analysis tracking L1/L2/L3 cache misses, RAM memory bandwidth limits, IPC (instructions per cycle), dynamic core frequency scaling, and thermal throttling.
Benchmark Automation & Tooling: Build reproducible, scriptable C++/Python benchmark suites to record inference latency, first-token time, peak memory footprint, and CPU utilization across varied batch sizes.
COMPENSATION:
Cash paid by hours billed for project work
Equity compensation is available for broader participation in DiffAI outside of an individual project
QUALIFICATIONS:
Ability to devote ~15 hours per week on avg.
Languages: Expert-level C++ and Python skills.
Systems Architecture: In-depth knowledge of CPU architecture (NUMA, L1-L3 cache hierarchies, memory channels, thread synchronization paradigms).
Frameworks & Libraries: Experience with CPU ML inference runtimes (ONNX Runtime, OpenVINO, Intel Extension for PyTorch/TensorFlow, C++ BLAS libraries like OpenBLAS/oneDNN).
Profiling Tools: Proficiency with Linux low-level performance tools (perf, Intel VTune Profiler, Valgrind/Cachegrind, or platform-specific telemetry tools).
Compilation & Hardware Extensions: Understanding of GCC/Clang flags, SIMD vectorization, and model quantization approaches.
Skills
Machine Learning
Experience
5+ yrs
Work arrangement
Yes
Skills from the listing
- Machine Learning
- Python
- C++
- Linux
- Deep Learning
- TensorFlow
- PyTorch
Your early-applicant advantage
How quickly JobBeeper finds and shares openings.
Job alerts
Get notified about new openings
Save your preferences, then add a paid plan for matching job alerts.