Notes

Sentiment analysis is one of the oldest tasks in natural language processing. In this note we will introduce some examples and terminology, some key problems in the field and a simple model that we can understand by just knowing Backpropagation Log Linear Models and the Softmax Function. We say: Polarity: the orientation of the sentiment. Subjectivity: if it expresses personal feelings. See demo Some applications: Businesses use sentiment analysis to understand if users are happy or not with their product. It’s linked to revenue: if the reviews are good, usually you make more money. But companies can’t read every review, so they want automatic methods. ...

Softmax is one of the most important functions for neural networks. It also has some interesting properties that we list here. This function is part of The Exponential Family, one can also see that the sigmoid function is a particular case of this softmax, just two variables. Sometimes this could be seen as a relaxation of the action potential inspired by neuroscience (See The Neuron for a little bit more about neurons). This is because we need differentiable, for gradient descent. The action potential is an all or nothing thing. ...

This is a quite good resource about this part of Support Vector Machines (step by step derivation). (Bishop 2006) chapter 7 is a good resource. The main idea about this supervised method is separating with a large gap. The thing is that we have a hyperplane, when this plane is projected to lower dimensional data, it can look like a non-linear separator. After we have found this separator, we can intuitively have an idea of confidence based on the distance of the separator. ...

This note extends the content Markov Processes in this specific context. Standard notions Explore-exploit dilemma 🟩 We have seen something similar also in Active Learning when we tried to model if we wanted to look elsewhere or go for the maximum value we have found. The dilemma under analysis is the explore-exploit dilemma: whether if we should just go for the best solution we have found at the moment, or look for a better one. This also has implications in many other fields, also in normal human life there are a lot of balances in these terms. ...

This is the generalization of the family of function where Softmax Function belongs. Many many functions are part of this family, most of the distributions that are used in science are part of the exponential family, e.g. beta, Gaussian, Bernoulli, Categorical distribution, Gamma, Beta, Poisson, are all part of the exponential family. The useful thing is the generalization power of this set of functions: if you prove something about this family, you prove it for every distribution that is part of this family. This family of functions is also closely linked too Generalized Linear Models (GLMs). ...

This note is still a TODO. Transliteration is learning learning a function to map strings in one character set to strings in another character set. The basic example is in multilingual applications, where it is needed to have the same string written in different languages. The goal is to develop a probabilistic model that can map strings from input vocabulary $\Sigma$ to an output vocabulary $\Omega$. We will extend the concepts presented in Automi e Regexp for Finite state automata to a weighted version. You will also need knowledge from Descrizione linguaggio for definitions of alphabets and strings, Kleene Star operations. ...

$$p(\theta \mid x_{1:n}, y_{1:n}) = \frac{1}{z} p(y_{1:n} \mid \theta, x_{1:n}) p(\theta \mid x_{1:n}) \approx q(\theta \mid \lambda)$$ For Bayesian Linear Regression we had high dimensional Gaussians which made the inference closed form, in general this is not true, so we need some kinds of approximation. Laplace approximation Introduction to the Idea 🟩 $$\psi(\theta) \approx \hat{\psi}(\theta) = \psi(\hat{\theta}) + (\theta-\hat{\theta} ) ^{T} \nabla \psi(\hat{\theta}) + \frac{1}{2} (\theta-\hat{\theta} ) ^{T} H_{\psi}(\hat{\theta})(\theta-\hat{\theta} ) = \psi(\hat{\theta}) + \frac{1}{2} (\theta-\hat{\theta} ) ^{T} H_{\psi}(\hat{\theta})(\theta-\hat{\theta} )$$ We simplified the term on the first order because we are considering the mode, so the gradient should be zero for the stationary point. ...

Introduction to Wide Column Storages One of the bottlenecks of traditional relational databases is the speed of the Joints, which could be done in $\mathcal{O}(n)$ using a merge join, assuming some indexes are present which make the keys already sorted. The other solution, of just using Distributed file systems, is also not optimal: they have usually a high latency, with high throughput, that is not optimal with the series of small files that it is optimized for. While Object Storages, do not have APIs that could be helpful -> richer logical model. ...

Introduzione Intuizione del campo elettrostatico Elettrostatico vs elettrodinamico 🟩 Andiamo a chiamare elettrostatico perché nel nostro caso non si sta muovendo nessuna carica all’itnerno di questo campo. Proprietà del campo elettrostatico (5) 🟨 Le linee di forza in ogni punto dello spazio sono tangenti e concorde al campo in quel punto; le linee di forza si addensano dove l’intensità del campo e maggiore; le linee di forza non si incrociano mai, in quanto in ogni punto il campo è definito univocamente e non può avere due direzioni distinte. le linee di forza hanno origine dalle cariche positive e terminano sul cariche negative; qualora ci siano solo cariche dello stesso segno le linee di forza si chiudono all’ infinito; nel caso di cariche di segno opposto, ma eguali in modulo, tutte le linee the partono dalle cariche positive si chiudono su quelle negative (induzione completa), alcune passando eventualmente per l’infinito; se invece le cariche non sono eguali in modulo, alcune linee terminano o provengono dall’ infinito. Carica esploratrice 🟩 È anche chiamata carica di prova, è una carica fittizia messa per esplorare la struttura del campo elettrico in un certo spazio ...

Gli argomenti della lezione 31 Ottobre sono circa da pagina 164 fino a 185 del mazzoldi. Leggi di Ohm Introduzione microscopica 🟩 Sappiamo che $$\vec{J} = -n e \vec{v}_{d} ne^{2} t \frac{\vec{E}}{m}$$ Vedi analisi della velocità di deriva col modello del 1900 in Corrente Elettrica. Dove abbiamo utilizzato la definizione di densità di corrente e la velocità fra collisioni ed altre Questo è una motivazione per considerare la densità di corrente come se fosse nello stesso verso. ...