Bias vs Variance

Bias

In a model, it is the systematic error that is cause due to wrong assumptions made during the learning process.

$${\text{Bias}}_D(\hat{Y})=E[\hat{Y}]-Y$$

Variance

It is the measure of the deviation of the data from it’s mean position.

$${\text{Var}}_D(\hat{Y})=E[(E[\hat{Y}]-\hat{Y}{)}^2]$$

where,

• $\hat{Y}\to$ Predicted target value
• $Y\to$ Actual target value
• $D\to$ Dataset

Reducing Bias

• Increase model complexity / Parameters.
• May cause over-fitting.

Reducing Variance

• Cross validation.
• Feature selection (selecting only necessary features reduces model complexity and variance error).
• Regularization.