Approximation through Regression

For given N Data points

$$\left(x_i,y_i\right),i=1,\dots ,N$$

We search for a Regression Curve with M basis polynomials

$$s(x)={\alpha }_1{s}_1(x)+{\alpha }_2{s}_2(x)+\dots +{\alpha }_M{s}_M(x)$$

Where $N\to$ is the number of data-points, $M\to$ points considered for regression

Quadratic Error

$$J(\mathbf{a})=\Vert \mathbf{y}-A\mathbf{a}{\Vert }_2^2$$

Where,

• Weights used to fit the regression model to the given function.

$$\mathbf{a}=\left(\begin{array}{c}{\alpha }_1\\ ⋮\\ {\alpha }_M\end{array}\right)\in {\mathbb{R}}^M$$

• Evaluation of the basis function at the given data points.

$$A=\left(\begin{array}{ccc}{s}_1\left(x_1\right)& \cdots & s_M\left(x_1\right)\\ ⋮& & ⋮\\ s_1\left(x_N\right)& \cdots & s_M\left(x_N\right)\end{array}\right)\in {\mathbb{R}}^{N\times M}$$

Note: The basis function for a polynomial regression is $s_i(x)=x^i$.  

• Is the dependent variable of the given data set.

$$\mathbf{y}=\left(\begin{array}{c}{y}_1\\ ⋮\\ y_N\end{array}\right)\in {\mathbb{R}}^N$$

230418 Regression