MAP estimation for a Gaussian distributed data model using regularization

To get the MAP parameters

1. MAP definition for Probabilistic Modeling

   $$w_{MAP}=\arg \underset{w}{\max }P(w|D)=\arg \underset{w}{\max }P(D|w)P(w)$$

2. For Gaussian case:

   $$w_{MAP}=\arg \underset{w}{\min }\left[\frac{1}{2}\sum _n(t_n-w^T\varphi (x_n){)}^2+\frac{\lambda }{2}||w|{|}^2\right]$$

3. Remember: This is just squared error + L2 regularization!

4. Linear model solution:

   $$w_{MAP}=(\lambda I+{\Phi }^T\Phi {)}^{-1}{\Phi }^{T}t$$

Key to remember: MAP = ML + regularization (prior acts as regularizer)

Note:

• Simple linear: $y(x,w)=w^{T}x=w_0+w_1{x}_1+\dots$
• Polynomial: $y(x,w)=w^T\varphi (x)$ where $\varphi (x)=\left[1,x,x^2,\dots \right]$
• RBF: $y(x,w)=w^T\varphi (x)$ where $\varphi (x)=\left[\exp \left(-{\Vert x-{\mu }_1\Vert }^2\right),\dots \right]$