Intuition

The most important observation that allows Fourier series approximation is that given $k = 1, 2, \dots$ we have that

$$ \frac{1}{\sqrt{ 2\pi }}, \frac{\cos(kx)}{\sqrt{ \pi }}, \frac{\sin(kx)}{\sqrt{ \pi }}, \dots $$

Form a infinitely dimensional orthonormal basis given the integral relations

$$ \int_{0}^{2\pi} (\sin (kx))^{2} \, dx = \int_{0}^{2\pi} (\cos(kx))^{2} \, dx = \pi $$ $$ \int_{0}^{2\pi}\sin(kx)\sin(hx) \, dx = \int_{0}^{2\pi}\cos(kx)\cos(hx) \, dx = 0 $$

And that

$$ \int_{0}^{2\pi}\sin(kx)\cos(hx) \, dx = \int_{0}^{2\pi} \sin(kx) \, dx = \int_{0}^{2\pi}\cos(hx) \, dx = 0 $$

Proofs of the relations

In this section we quickly prove why the above equations hold. First we all agree that $\int_{0}^{2\pi} \sin(kx) \, dx = \int_{0}^{2\pi} \cos(hx) \, dx = 0$ because their period divides $2\pi$ and the sum of the area of a period is clearly 0. Or we can explicitly find the primitive and solve

$$ \int_{0}^{2\pi} \sin(kx) \, dx = -\frac{1}{k} \cos(kx) \bigg\vert_{0}^{2\pi} = 0 - 0 $$

Equivalently the other part with the cosine.

For the other relations we need to remember some trigonometric identities:

$\sin(kx)\cos(hx) = \frac{1}{2}\left[ \sin(kx + hx) + \sin(kx - hx) \right]$ And if we solve now the integral we can see that

$$ \int_{0}^{2\pi} \sin(kx) \cos(hx) \, dx = \frac{1}{2} \left[ \int_{0}^{2\pi} \sin((k + h)x) \, d + \int_{0}^{2\pi}\sin((k - h)x) \, dx \right] $$ $$ = = \frac{1}{2}\left[ \frac{-1}{k + h} \cos((k + h)x)\bigg\vert_{0}^{2\pi} + \frac{-1}{k - h} \cos((k - h)x) \bigg\vert_{0}^{2\pi} \right] = \frac{1}{2} (0 + 0) $$

Same thing with the others but we use the identities

$$ \sin x \cdot \sin y = \frac{1}{2}\left[ \cos(x - y) - \cos(x + y) \right] $$

And

$$ \cos x \cdot \cos y = \frac{1}{2} \left[ \cos(x - y) + \cos(x + y) \right] $$

And then you can prove every relation.

Fourier Series

This allows us to define the projection $S_{n}$ of every function $f$ onto that basis. Defined as

$$ S_{n}f = \sum_{i=0}^{2n} \langle f, e_{i} \rangle e_{i} $$

Which is explicitely:

$$ S_{n}f(x) = \left( \int_{0}^{2n} \frac{1}{\sqrt{ 2\pi }} f(t) \, dt \right) \frac{1}{\sqrt{ 2\pi }} + \sum_{k=1}^{n} \left[ \left( \int_{0}^{2n} \frac{\cos(kt)}{\sqrt{ \pi }} f(t) \, dt \right) \frac{\cos(kx)}{\sqrt{ \pi }} + \left( \int_{0}^{2\pi} \frac{\sin(kt)}{\sqrt{ \pi }}f(t) \, dt \right) \frac{\sin(kx)}{\sqrt{ \pi }} \right] $$

Which could be rewritten as follows (called Fourier Series):

$$ S_{n}f(x) = \frac{1}{2}a_{0} + \sum_{k=1}^{n}(a_{k}\cos(kx) + b_{k}\sin(kx)) $$

With $a_{k} = \frac{1}{\pi}\int_{0}^{2\pi} f(t) \cos (kt) \, dt$ and $b$ equivalently. these $a, b$ are called Fourier coefficients of $f$. We call $S_{n}f(x)$ nth Fourier sum

The above series can be written equivalently in:

$$ S_{n}f(x) = \sum_{n = -N}^{N}C_{n}e^{i 2\pi xn/p} $$

Properties of the Fourier Series

Norm of the Fourier Series

We assert that

$$ \lVert S_{n} f \rVert ^{2} = \pi \left[ \frac{a_{0}^{2}}{2} + \sum_{k = 1}^{n} (a_{k}^{2} + b_{k}^{2}) \right] $$

In this case the definition of the norm squared is as follows:

$$ \lVert S_{n}f \rVert^{2} = \int_{0}^{2\pi} \lvert S_{n}f \rvert ^{2} \, dx $$

And then it is far easier to derive.

The Fourier transform

We used this to solve a differential equation in Diffusion Models.

The transformations

the Fourier transform of the function $f(x, t)$ is

$$ F(k, t) = \int e^{-ikx}f(x, t) \, dx $$

And it’s inverse is

$$ f(x, t) = \int e^{ikx}F(k, t) \, dk $$