Hi! I’m Rohan. I’m an engineer currently at Meta AI. Previously, I graduated with a BS in Computer Science from UCLA. I grew up in San Jose, California.


I currently work at Meta AI on enabling large-scale distributed training within the PyTorch framework. You can find more about my research interests at this link and some publications at this link

Previously, I’ve worked on building mortgage lending products at Blend, risk models at Affirm, and messaging applications for sports teams at Hudl.


This blog is mostly a place for me to collect random notes and tips that I’ve found interesting and useful over the years. Below is a reverse-chronological set of links to some things that I’ve written or helped write over the years!


Mitigating stragglers in large-scale jobs with PyTorch DDP


A tutorial on using PyTorch FSDP for large-scale deep learning training

Scaling foundation multimodal models using PyTorch FSDP

PyTorch Data Parallel on GCP

Scaling to 1 trillion parameters with PyTorch FSDP


How to use PyTorch’s RPC Framework


Do One Thing

Modern Addiction


My Internship at Affirm: Crafting a Reliable Metrics and Alerting Framework


Welcome to AI at UCLA

Getting Started in Machine Learning


This section is still in progress!

Contact me

rvarm1 @ gmail dot com