MAP solution

A MAP estimation with a Gaussian prior on the weights and a Gaussian likelihood on the data is equivalent to minimizing the sum of squared errors with L2 regularization.

Formula:

Key Insight:

The regularization strength is inversely proportional to the variance of the Gaussian prior :

  • Strong Regularization (large ) Small Prior Variance (weights are assumed to be near zero).
  • Weak Regularization (small ) Large Prior Variance (weights are allowed to vary more).

Key Point: The model is linear in parameters but can be non-linear in inputs through the choice of basis functions .