Normalizing flows have both latent space and can produce tractable explicit probability distributions (closer to Autoregressive Modelling , they have tractable distributions, but not a latent space). This means we are able to get the likelihoods of a certain sample. This…
On Autoregressivity # The main idea of autoregressivity is to use previous prediction to predict the next state. The Autoregressive property # Autoregressive models model a joint distribution of aleatoric variables by assuming a chain rule like decomposition: p ( x ) = i = 1 ∏ n…
Introduction: a neuron # I am lazy, so I'm skipping the introduction for this set of notes. Look at Andrew Ng's Coursera course for this part ( here are the notes). Historical paper is (Rosenblatt 1958) . One can view a perceptron to be a Log Linear Models with the temperature…
The perceptron is a fundamental binary linear classifier introduced by (Rosenblatt 1958) . It maps an input vector x ∈ R n to an output y ∈ { 0 , 1 } using a weighted sum followed by a threshold function. Introduction to the Perceptron # A mathematical model # Given an input…
3D representations # In this section, we present some of the most common 3D representations used in computer graphics and computer vision. Each representation has its own advantages and disadvantages, and the choice of representation often depends on the specific application.…
Recurrent Neural Networks allows us to model arbitrarily long sequence dependencies, at least in theory (this is also why they seem a very nice choice in theory for time series). This is very handy, and has many interesting theoretical implication. But here we are also…
Backpropagation is perhaps the most important algorithm of the 21st century. It is used everywhere in machine learning and is also connected to computing marginal distributions. This is why all machine learning scientists and data scientists should understand this algorithm very…
Transformers, introduced in NLP language translation in (Vaswani et al. 2017) , are one of the cornerstones of modern deep learning. For this reason, it is quite important to understand how they are done. Introduction to Transformers # Transformers are called in this manner…
Introduction to Convolutional NN # Design Goals # We want to be invariant to some transformations but also at the same time to be specific to some thing. Convolutional Neural Networks (CNNs) are a class of deep neural networks that are particularly effective for image processing…
Egocentric vision is a sub-field of computer vision that studies vision understanding from a centered point of view, that typical of animals. One historical thing is MIT 1997 they had to bring around very heavy cameras. Now we have glasses. Other examples of egocentric vision…