MAP solution
A MAP estimation with a Gaussian prior on the weights and a Gaussian likelihood on the data is equivalent to minimizing the sum of squared errors with L2 regularization.
Formula:
Key Insight:
The regularization strength is inversely proportional to the variance of the Gaussian prior :
- Strong Regularization (large ) Small Prior Variance (weights are assumed to be near zero).
- Weak Regularization (small ) Large Prior Variance (weights are allowed to vary more).
Key Point: The model is linear in parameters but can be non-linear in inputs through the choice of basis functions .