Hi! I’m Rohan. I’m an engineer currently at Meta AI. Previously, I graduated with a BS in Computer Science from UCLA. I grew up in San Jose, California.
Work
I currently work at Meta AI on enabling large-scale distributed training within the PyTorch framework. You can find more about my research interests at this link and some publications at this link
Previously, I’ve worked on building mortgage lending products at Blend, risk models at Affirm, and messaging applications for sports teams at Hudl.
Writing
This blog is mostly a place for me to collect random notes and tips that I’ve found interesting and useful over the years. Below is a reverse-chronological set of links to some things that I’ve written or helped write over the years!
2023
Mitigating stragglers in large-scale jobs with PyTorch DDP
2022
A tutorial on using PyTorch FSDP for large-scale deep learning training
Scaling foundation multimodal models using PyTorch FSDP
Scaling to 1 trillion parameters with PyTorch FSDP
2020
How to use PyTorch’s RPC Framework
2019
2018
My Internship at Affirm: Crafting a Reliable Metrics and Alerting Framework
2017
Getting Started in Machine Learning
Misc
This section is still in progress!
Contact me
rvarm1 @ gmail dot com