I train and post-train large language models, and build the infrastructure that makes reinforcement learning on them work. I led post-training for Llama 3.0 and the team behind Llama Guard, and co-created OpenEnv, an open standard for RL environments.
I write here for ML engineers who know supervised learning well and want the rest of the map.
Writing
- A primer on LLM post-training PyTorch blog
- Differential privacy, part 3: per-sample gradients for more layers PyTorch on Medium
- Differential privacy, part 2: efficient per-sample gradients in Opacus PyTorch on Medium
- Differential privacy, part 1: DP-SGD explained PyTorch on Medium
Talks
Watch- Reinforcement learning, agents, and OpenEnv GPU MODE, RL Mini Conference
- Differential privacy on PyTorch PyTorch Developer Day
- Differentially private model training with Opacus OpenMined PriCon
Selected publications
All publications- LogAct: Enabling Agentic Reliability via Shared LogsBalakrishnan, Bharambe, Testuggine, et al. arXiv.
- Introducing v0.5 of the AI Safety Benchmark from MLCommonsVidgen et al. Meta representative on the working group. arXiv.
- The Llama 3 Herd of ModelsLlama Team. Contributor. arXiv.
- Llama Guard: LLM-based Input-Output Safeguard for Human-AI ConversationsInan, Upasani, Chi, …, Testuggine, Khabsa. arXiv.