About

Hi! I’m Rohan. I’m an engineer currently at Meta AI. Previously, I graduated with a BS in Computer Science from UCLA. I grew up in San Jose, California.

Work

I currently work at Meta AI on enabling large-scale distributed training within the PyTorch framework. You can find more about my research interests at this link and some publications at this link

Previously, I’ve worked on building mortgage lending products at Blend, risk models at Affirm, and messaging applications for sports teams at Hudl.

Writing

This blog is mostly a place for me to collect random notes and tips that I’ve found interesting and useful over the years. Below is a reverse-chronological set of links to some things that I’ve written or helped write over the years!

2023

Mitigating stragglers in large-scale jobs with PyTorch DDP

2022

A tutorial on using PyTorch FSDP for large-scale deep learning training

Scaling foundation multimodal models using PyTorch FSDP

PyTorch Data Parallel on GCP

Scaling to 1 trillion parameters with PyTorch FSDP

2020

How to use PyTorch’s RPC Framework

2019

Do One Thing

Modern Addiction

2018

My Internship at Affirm: Crafting a Reliable Metrics and Alerting Framework

2017

Welcome to AI at UCLA

Getting Started in Machine Learning

Misc

This section is still in progress!

Contact me

rvarm1 @ gmail dot com