Squared Error Function

$$E(w)=\frac{1}{2}\sum _{n=1}^N{\left(y\left(x_n,w\right)-t_n\right)}^2$$

Remember:

• $1/2$ factor (makes derivative cleaner)
• Sum over all $N$ data points
• Squared difference between:
  • $y\left(x_n,w\right)=$ model prediction
  • $t_n=$ target/actual value

Mnemonic: “Half the Sum of Squared differences” Note: Sometimes written as $\frac{1}{2N}$ for averaging

Generalized Squared Error

$$\tilde{E}(\mathbf{w})=\frac{1}{2}\sum _n(t_n-{\mathbf{w}}^T\varphi (x_n){)}^2+\frac{\lambda }{2}||\mathbf{w}|{|}^2$$

• $\mathbf{w}$ is the parameter vector (weights) that the model learns (and optimized during training).
• $\varphi (x)$ is the Basis Function vector which Transforms the raw input x into a higher-dimensional feature space
• Transforms the raw input $x$ into a higher-dimensional feature space
• The dot product $w^T\varphi (x)$ gives the model’s prediction