These are some machine learning notes, thoughts, tips & tricks that I’ve collected throughout the years of my studies. Some of them are from Introduction to Machine Learning and Neural Networks and Deep Learning taken at UCLA. Others are from various online courses such as Stanford’s CS231n, or books such as the deep learning book.

These notes represent my own understanding of the material and are likely to contain some errors. If you find any errors, inaccuracies, or points that could be clarified better, please send me a pull request or contact me!

Linear and Logistic Regression

• Probabalistic Interpretation for Linear Regression & Gradient Descent
• Linear & Logistic Regression, Convexity, Hessian Matrices, and closed-form solutions
• Regularized (ridge) linear regression

Regularization & Preventing Overfitting

• L1 and L2 regularization basics

Support Vector Machines

• SVM, Dual & Primal formation, Optimization with Lagrangians

Kernel Methods

• Kernel Methods, kernelized ridge regression
• Kernelized SVMs

Convolutional Networks

• Convolutional Networks, Filter Sizes
• Notes on ResNets

NLP/RNNs/LSTMs/Language Modeling

• Sequence to Sequence Paper notes
• Attention paper notes

Probability and Linear Algebra

• Linear Algebra primer

Misc

• Xavier initialization of weights
• Downsides of sigmoid function, and why inputs should be centered