Finding Hyper Parameters

Steps involved in selecting the Hyper Parameters

1. Split Data
   • The available data is split into subsets namely the Training Set, Validation Set and the Test Set.
2. Training
   • Here the model parameters are learned.
3. Validation
   • The model is then parameterized with different Hyper-parameters $\lambda$. This is also known as tuning the model.
4. Test Set
   • Finally the model is tested to evaluate it’s performance. Here the Generalization error can be calculated (How well the model deals with unseen data.)

Note: The Hyper Parameters may be the Learning Rate $(\eta )$, Regularisation parameter $(\lambda )$ etc.