Fisher's Linear Discriminant Function

Motivation

We try to find a weight vector $\mathbf{w}$ such that the Hyper Plane is $\perp$ to it.

The position of the Hyper Plane however is not known. We can find it using the following methods.

Approach 1: Maximize the inter-class variance

Maximise the distance between the projected means. In simple terms we try to find $\mathbf{w}$ that has the greatest distance between the projected means. Mathematically

$$\begin{array}{rl}\underset{\mathbf{w}}{\max }\left|m_1^{\prime }-m_2^{\prime }\right|& \iff \underset{\mathbf{w}}{\max }\left|{\mathbf{w}}^T{\mathbf{m}}_1-{\mathbf{w}}^T{\mathbf{m}}_2\right|\\ & =\underset{\mathbf{w}}{\max }\left|{\mathbf{w}}^T({\mathbf{m}}_1-{\mathbf{m}}_2)\right|\end{array}$$

Hence,

$$\mathbf{w}\propto \left({\mathbf{m}}_2-{\mathbf{m}}_1\right)$$

$\mathbf{w}$ points in the same direction as $({\mathbf{m}}_2-{\mathbf{m}}_1)$

source