Fourier Series

Intuition

The most important observation that allows Fourier series approximation is that given $k=1,2,\dots$ we have that

$$\frac{1}{\sqrt{2\pi }},\frac{\cos (kx)}{\sqrt{\pi }},\frac{\sin (kx)}{\sqrt{\pi }},\dots$$

Form a infinitely dimensional orthonormal basis given the integral relations

$${\int }_0^{2\pi }(\sin (kx){)}^{2}dx={\int }_0^{2\pi }(\cos (kx){)}^{2}dx=\pi$$ $${\int }_0^{2\pi }\sin (kx)\sin (hx)dx={\int }_0^{2\pi }\cos (kx)\cos (hx)dx=0$$

And that

$${\int }_0^{2\pi }\sin (kx)\cos (hx)dx={\int }_0^{2\pi }\sin (kx)dx={\int }_0^{2\pi }\cos (hx)dx=0$$

Proofs of the relations

In this section we quickly prove why the above equations hold. First we all agree that ${\int }_0^{2\pi }\sin (kx)dx={\int }_0^{2\pi }\cos (hx)dx=0$ because their period divides $2\pi$ and the sum of the area of a period is clearly 0. Or we can explicitly find the primitive and solve

$${\int }_0^{2\pi }\sin (kx)dx=-\frac{1}{k}\cos (kx){|}_0^{2\pi }=0-0$$

Equivalently the other part with the cosine.

For the other relations we need to remember some trigonometric identities:

$\sin (kx)\cos (hx)=\frac{1}{2}\left[\sin (kx+hx)+\sin (kx-hx)\right]$ And if we solve now the integral we can see that

$${\int }_0^{2\pi }\sin (kx)\cos (hx)dx=\frac{1}{2}\left[{\int }_0^{2\pi }\sin ((k+h)x)d+{\int }_0^{2\pi }\sin ((k-h)x)dx\right]$$ $$==\frac{1}{2}\left[\frac{-1}{k+h}\cos ((k+h)x){|}_0^{2\pi }+\frac{-1}{k-h}\cos ((k-h)x){|}_0^{2\pi }\right]=\frac{1}{2}(0+0)$$

Same thing with the others but we use the identities

$$\sin x\cdot \sin y=\frac{1}{2}\left[\cos (x-y)-\cos (x+y)\right]$$

And

$$\cos x\cdot \cos y=\frac{1}{2}\left[\cos (x-y)+\cos (x+y)\right]$$

And then you can prove every relation.

Fourier Series

This allows us to define the projection $S_n$ of every function $f$ onto that basis. Defined as

$$S_{n}f=\sum _{i=0}^{2n}⟨f,e_i⟩e_i$$

Which is explicitely:

$$S_{n}f(x)=\left({\int }_0^{2n}\frac{1}{\sqrt{2\pi }}f(t)dt\right)\frac{1}{\sqrt{2\pi }}+\sum _{k=1}^n\left[\left({\int }_0^{2n}\frac{\cos (kt)}{\sqrt{\pi }}f(t)dt\right)\frac{\cos (kx)}{\sqrt{\pi }}+\left({\int }_0^{2\pi }\frac{\sin (kt)}{\sqrt{\pi }}f(t)dt\right)\frac{\sin (kx)}{\sqrt{\pi }}\right]$$

Which could be rewritten as follows (called Fourier Series):

$$S_{n}f(x)=\frac{1}{2}{a}_0+\sum _{k=1}^n(a_k\cos (kx)+b_k\sin (kx))$$

With $a_k=\frac{1}{\pi }{\int }_0^{2\pi }f(t)\cos (kt)dt$ and $b$ equivalently. these $a,b$ are called Fourier coefficients of $f$. We call $S_{n}f(x)$ nth Fourier sum

The above series can be written equivalently in:

$$S_{n}f(x)=\sum _{n=-N}^N{C}_n{e}^{i2\pi xn/p}$$

Properties of the Fourier Series

Norm of the Fourier Series

We assert that

$$\Vert S_{n}f{\Vert }^2=\pi \left[\frac{a_0^2}{2}+\sum _{k=1}^n(a_k^2+b_k^2)\right]$$

In this case the definition of the norm squared is as follows:

$$\Vert S_{n}f{\Vert }^2={\int }_0^{2\pi }|S_{n}f{|}^{2}dx$$

And then it is far easier to derive.

The Fourier transform

We used this to solve a differential equation in Diffusion Models.

The transformations

the Fourier transform of the function $f(x,t)$ is

$$F(k,t)=\int e^{-i2\pi kx}f(x,t)dx$$

And it's inverse is

$$f(x,t)=\int e^{i2\pi kx}F(k,t)dk$$