Hi! I’m Rohan. I’m an engineer currently at Meta AI. Previously, I graduated with a BS in Computer Science from UCLA. I grew up in San Jose, California.
I currently work at Meta AI on enabling large-scale distributed training within the PyTorch framework. You can find more about my research interests at this link and some publications at this link
Previously, I’ve worked on building mortgage lending products at Blend, risk models at Affirm, and messaging applications for sports teams at Hudl.
This blog is mostly a place for me to collect random notes and tips that I’ve found interesting and useful over the years. Below is a reverse-chronological set of links to some things that I’ve written or helped write over the years!
Mitigating stragglers in large-scale jobs with PyTorch DDP
A tutorial on using PyTorch FSDP for large-scale deep learning training
Scaling foundation multimodal models using PyTorch FSDP
PyTorch Data Parallel on GCP
Scaling to 1 trillion parameters with PyTorch FSDP
How to use PyTorch’s RPC Framework
Do One Thing
My Internship at Affirm: Crafting a Reliable Metrics and Alerting Framework
Welcome to AI at UCLA
Getting Started in Machine Learning
This section is still in progress!
rvarm1 @ gmail dot com