Bias vs Variance

Bias

In a model, it is the systematic error that is cause due to wrong assumptions made during the learning process.

$${\text{Bias}}_D(\hat{Y})=E[\hat{Y}]-Y$$

Variance

It is the measure of the deviation of the data from it’s mean position.

$${\text{Var}}_D(\hat{Y})=E[(E[\hat{Y}]-\hat{Y}{)}^2]$$

where,

• $\hat{Y}\to$ Predicted target value
• $Y\to$ Actual target value
• $D\to$ Dataset

To Reduce HIGH BIAS (Underfitting):

• Increase model complexity
• Add more features
• Reduce regularization (smaller λ)

To Reduce HIGH VARIANCE (Overfitting):

• Reduce model complexity
• More training data
• Regularization (L1/L2)
• Early stopping
• Cross-validation