Notes

An historical perspective The origins of motion capture One of the earliest starts of motion capturing is the famous horse in 1878 in motion “video”. This was the start of all the modern cameras. One of the earliest human body motion capture was in military for moving efficiency purposes in 1883. This website has many historical resources on the topic. The problem is still a problem in modern times. If we want to create models to mimic humans, it surely could be nice to understand how humans move and think. This is the general line of though of this line of research. ...

Recurrent Neural Networks allows us to model arbitrarily long sequence dependencies, at least in theory (this is also why they seem a very nice choice in theory for time series). This is very handy, and has many interesting theoretical implication. But here we are also interested in the practical applicability, so we may need to analyze common architectures used to implement these models, the main limitation and drawbacks, the nice properties and some applications. ...

The Skylake processor is a 2015 Intel processor. The Intel Processor In 1978 Intel made the choice to have retrocompatibility for every processor. At that time they had the 8086 processor, with some number of memory bits. For backwards compatibility intructions have usually just grown. They used geographic locations because these are not suable. If we want new code to run for old processors, we should need to put specific flags. ...

The perceptron is a fundamental binary linear classifier introduced by (Rosenblatt 1958). It maps an input vector $\mathbf{x} \in \mathbb{R}^n$ to an output $y \in \{0,1\}$ using a weighted sum followed by a threshold function. Introduction to the Perceptron A mathematical model Given an input vector $\mathbf{x} = (x_1, x_2, \dots, x_n)$ and a weight vector $\mathbf{w} = (w_1, w_2, \dots, w_n)$, the perceptron computes: $$ z = \mathbf{w}^\top \mathbf{x} + b = \sum_{i=1}^{n} w_i x_i + b $$$$ y = f(z) = \begin{cases} 1, & \text{if } z \geq 0 \\ 0, & \text{otherwise} \end{cases} $$Learning Rule Given a labeled dataset $\{ (\mathbf{x}^{(i)}, y^{(i)}) \}_{i=1}^{m}$, the perceptron uses the following weight update rule for misclassified samples ($y^{(i)} \neq f(\mathbf{w}^\top \mathbf{x}^{(i)} + b)$): ...

Transformers, introduced in NLP language translation in (Vaswani et al. 2017), are one of the cornerstones of modern deep learning. For this reason, it is quite important to understand how they are done. Introduction to Transformers Transformers are called in this manner because they transform the input data space into another with the same dimensionality. The goal of the transformation is that the new space will have a richer internal representation that is better suited to solving downstream tasks. (Bishop & Bishop 2024) ...

This note is one of the few notes that was generated with the help of chatgpt. Bayesian Information Criterion (BIC) The Bayesian Information Criterion (BIC) is a model selection criterion that helps compare different statistical models while penalizing model complexity. It is rooted in Bayesian probability theory but is commonly used even in frequentist settings. Mathematically Precise Definition For a statistical model $M$ with $k$ parameters fitted to a dataset $\mathcal{D} = \{x_1, x_2, \dots, x_n\}$, the BIC is defined as: ...

Budget A definition for Budget Economist want simple models to start to model things. One of the things we will model here is how do you describe what you can afford about some goods. Budget Set $$ \text{Budget Constraint inequality}:p^{T}x \leq m $$ We define Budget Set to be $P_{p, m} = \left\{ x \in \mathbb{R}^{d} : p^{T}x \leq m\right\}$ Composite Goods $$ p_{1}x_{1} + x_{2} \leq m $$ Where $x_{2} = \sum_{i = 2}^{n} p_{i}x_{i}$. In this case, $x_{2}$ is called composite good, which is a sort of abstraction to simplify some calculus. ...

Here we analyze how demand changes when prices and income changes. Types of Goods Here we will define two main types of Goods: Normal Goods: The demand increases linearly with the income. Inferior or Ordinary Goods: The demand decreases when the income is higher, one example is low quality food. Necessary Good: The demand is increases sublinearly (e.g. constantly) with respect to the income. Luxury Good: The demand increases more than linearly with the income. Giffen Good: The decrease in price leads to a decrease in demand. Curves of Demand Income offer curve It is quite intuitive to visualize: just expand the income while keeping utility and priced fixed, you will see the optimal point translating along the curve. ...

In this note we will first talk about briefly some of the main differences of the three main approaches regarding statistics: the bayesian, the frequentist and the statistical learning methods and then present the concept of the estimator, compare how the approaches differ from method to method, we will explain maximum likelihood estimator and the Rao-Cramer Bound. Short introduction to the statistical methods Bayesian $$ p(\theta \mid X) = \frac{1}{z}p(X \mid \theta) p(\theta) $$The quantity $P(X \mid \theta)$ could be very complicated if our model is complicated. ...

Let’s consider first a simple model for apartments in a college. Here we are interested to predict the prices of the rooms, and how we can allocate them to students. For simplicity, we will assume that they are all equal except for the location, which could be inner or outer. Types of variables Economist will say that parameters for a model, i.e. variables that are fixed for some type of analysis exogenous variables, while the variables of interest of a model endogenous variables. In this setting, it could be the location for a certain room. ...