Parameter Estimation for Maximum Likelihood (Gaussian Mixture Model)

A GMM is a sum of multiple weighted Gaussian components:

$$p(x|\theta )\sim \sum _{k=1}^K{c}_k\cdot \mathcal{N}(x;{\mu }_k,{\Sigma }_k)$$

When trying to find the ML estimates for all the weights ($c_k$), means (${\mu }_k$), and covariances (${\Sigma }_k$), the set of equations derived from setting the partial derivatives of the log-likelihood to zero cannot be solved directly.

This is because of the sum inside the logarithm, which makes a closed-form solution intractable. Instead, iterative methods like the Expectation-Maximization (EM) algorithm* are required.