Public listing

Machine Learning & CPU Performance Engineer (Senior)

Differential AI United States (Remote)
  • Experience5+ yrs
  • Work modeYes
  • ListedPosted 8 hours ago
  • SourceLinkedin

Role overview

About the role

Description supplied by the original job listing.

ABOUT US:
Differential AI (DiffAI) is an Ivy League PhD-inspired AI/ML startup that focuses on efficient, secure knowledge retrieval using smaller, highly optimized models that significantly reduce training and inference costs. Through our proprietary training and inference techniques, the platform offers high-accuracy, edge-ready AI solutions that can be deployed entirely on-premise or in single-tenant environments. Founded on principles derived from brand new PhD research, our technology offers compelling value propositions:
Private & Secure: Models run entirely offline, ensuring complete data privacy
Ultra-Efficient: 7,000x smaller models than conventional solutions
Cost-Effective: Up to 83% reduction in training costs
We are a post-revenue company with 2 patents pending. We are partnered with three F500 companies to deliver AI research projects. We are purposely bootstrapped after turning down a few investor indications. Our founding team consists of 2 AI/ML/Data Science PhDs and 1 MBA, with advanced degrees from Brown, Harvard, and Georgetown. We operate fully remotely.
YOUR RESPONSIBILITIES:
We are seeking someone to join our team in conducting systematic research on machine learning execution across CPU architectures. In this role, you will lead the implementation, runtime optimization, and micro-profiling of diverse ML model architectures purely on CPU backends (x86, ARM, multi-core server processors). You will build automated benchmarking tools to measure trade-offs between accuracy, latency, thread efficiency, memory bandwidth, and cache behavior.
CPU-Specific Runtime Optimization: Compile and run deep learning architectures using CPU-optimized runtimes (OpenVINO, ONNX Runtime, PyTorch CPU Inductor, llama.cpp / ggml, C++ implementations).
Thread & Memory Control: Configure and profile NUMA node topology, CPU core affinity, thread pinning, and thread pools (OpenMP, TBB) to isolate system variance and maximize throughput.
Instruction-Level Benchmarking: Profile and exploit low-precision CPU instructions (AVX-512, Intel AMX, ARM NEON/SVE) to evaluate low-bit quantization (INT8, FP8, BF16) versus floating-point precision on accuracy and speed.
Hardware Telemetry & Profiling: Conduct deep bottleneck analysis tracking L1/L2/L3 cache misses, RAM memory bandwidth limits, IPC (instructions per cycle), dynamic core frequency scaling, and thermal throttling.
Benchmark Automation & Tooling: Build reproducible, scriptable C++/Python benchmark suites to record inference latency, first-token time, peak memory footprint, and CPU utilization across varied batch sizes.
COMPENSATION:
Cash paid by hours billed for project work
Equity compensation is available for broader participation in DiffAI outside of an individual project
QUALIFICATIONS:
Ability to devote ~15 hours per week on avg.
Languages: Expert-level C++ and Python skills.
Systems Architecture: In-depth knowledge of CPU architecture (NUMA, L1-L3 cache hierarchies, memory channels, thread synchronization paradigms).
Frameworks & Libraries: Experience with CPU ML inference runtimes (ONNX Runtime, OpenVINO, Intel Extension for PyTorch/TensorFlow, C++ BLAS libraries like OpenBLAS/oneDNN).
Profiling Tools: Proficiency with Linux low-level performance tools (⁠perf⁠, Intel VTune Profiler, Valgrind/Cachegrind, or platform-specific telemetry tools).
Compilation & Hardware Extensions: Understanding of GCC/Clang flags, SIMD vectorization, and model quantization approaches.

Skills
Machine Learning

Experience
5+ yrs

Work arrangement
Yes

Skills from the listing

  • Machine Learning
  • Python
  • C++
  • Linux
  • Deep Learning
  • TensorFlow
  • PyTorch

Your early-applicant advantage

How quickly JobBeeper finds and shares openings.

9 min from Linkedin publishing this role to us finding it
11 min median time from a role going live to a subscriber being told
6 hours subscribers had this role before this page existed
68,521 roles found in the last 24 hours
See available plans →

Job alerts

Get notified about new openings

Save your preferences, then add a paid plan for matching job alerts.

Create free account
Log in to apply