GPU Performance Engineer

About the team

Adaptive ML is helping companies build singular generative AI experiences by democratizing the use of reinforcement learning. We are building the foundational technologies, tools, and products required for models to learn directly from users' interactions and for models to self-critique and self-improve from simple guidelines. Our tightly-knit team was previously involved in the creation of state-of-the-art open-access large language models. We have closed a $20M seed with Index & ICONIQ in early 2024, and are now live with our first enterprise customers.

Our Technical Staff is responsible for building the foundational technology powering Adaptive ML, in line with requests and requirements identified by our Product and Commercial Staff. We strive to build excellent, robust, and efficient technology, and to conduct at-scale, honest research with high-impact for our roadmap and customers.

About the role

As a GPU Performance Engineer in our Technical Staff, you will help ensure that our LLM stack (Adaptive Harmony) delivers state of the art performance across a wide variety of settings; from latency-bound regimes where serving requests with sub-second response times is key, to throughput-bound regimes during training and offline inference. You will help build the foundational technology powering Adaptive ML by delivering performance improvements directly to our clients as well as to our internal workloads.

Some examples of tasks you will encounter during your work:

• Profile and iterate GPU inference kernels in Triton or CUDA, identifying memory bottlenecks and optimizing latency—and decide how to adequately benchmark an inference service;
• Systematically identify and eliminate synchronization points between the CPU and GPU, enabling asynchronous communication of results from Python workers to our Rust backend;
• Develop and deploy complex high-performance systems, from stateful K,V-cache to test time search;
• Work with quantization methods to minimize the memory footprint of our models;
• Modify existing implementation of kernels to support requested features, and efficiently implement novel operations entirely from scratch.

We are looking for self-driven, intense individuals, who value technical excellency, honesty, and growth.

Your responsibilities

• Build and maintain fast and robust GPU code, focusing on delivering performance improvements in real world applications;
• Write high-quality software in CUDA, CUTLASS, or Triton with a focus on performance and robustness;
• Profile dedicated GPU kernels, optimizing across latency/compute-bound regimes for complex workloads;
• Contribute to our product roadmap, by identifying promising trends that can improve performance;
• Report clearly on your work to a distributed collaborative team, with a bias for asynchronous written communication.

Your (ideal) background

The background below is only suggestive of a few pointers we believe could be relevant; we welcome applications from candidates with diverse backgrounds, do not hesitate to get in touch if you think you could be a great fit even if the below doesn't fully describe you.

• A M.Sc./Ph.D. in computer science, or demonstrated experience in software engineering, preferably with a focus on GPU-optimization;
• Strong programming skills, preferably with a focus on systems and general purpose GPU programming;
• A track record of writing high performance kernels, having preferably demonstrated ability to reach state of the art performance on well defined tasks;
• Contributions to relevant open-source projects, such as CUTLASS, Triton and MLIR;
• Passionate about the future of generative AI, and eager to build foundational technology to help machines deliver more singular experiences.

Benefits

• Comprehensive medical (health, dental, and vision) insurance;
• 401(k) plan with 4% matching (or equivalent);
• Unlimited PTO — we strongly encourage at least 5 weeks each year;
• Mental health, wellness, and personal development stipends;
• Visa sponsorship if you wish to relocate to New York or Paris.

‍