Pattern Recognition

I. Bayesian Decision Theory

Fundamentals

1. Posterior, Likelihood, Prior and Evidence
2. Bayes Theorem

Bayesian Classification

1. Bayesian Classifier
2. Bayes Decision Rule or Maximum A Posteriori
3. Reconstructing Evidence
4. Classifier with the Lowest Error Probability
5. Error Rate comparison between a Random Guess and a Bayesian Guess

Maximum Likelihood and Related Concepts

1. Maximum Likelihood Decision Rule
2. Likelihood Ratio
   • Log Likelihood Ratio
   • Comparison of Likelihood Ratio between Max Likelihood and Bayes Decisions
3. Maximum Likelihood vs Maximum A Posteriori in action

Error and Risk Analysis

1. Probability Error
2. Bayes Risk

Discriminant Functions

1. Bayesian Discriminant Function
2. Equivalent Discriminant Function and Examples
3. Discriminant Function for Minimum Error Rate Classification
4. Discriminant Function (Multi-category)
5. Bayesian Discriminant Function (Gaussian)

Bayesian Networks

1. Bayesian Belief Networks
2. Computation of Variables in a Bayesian Belief Network

II. Quality Measures and Evaluation Metrics

Classification Metrics

1. Binary Classifications
2. Precision, Accuracy & Error Rate
3. Intersection over Union (IoU)

Performance Evaluation

1. Receiving Operator Characteristics (ROC)
2. Equal Error Rate

III. Supervised Estimation of Distribution Parameters

Probabilistic Models

1. Multivariate Gaussian Model and Gaussian Mixture Model
2. Total Parameters in a Gaussian Mixture Model

Parameter Estimation Techniques

1. Parameter Estimation for Maximum Likelihood
2. Parameter Estimation for Maximum Likelihood (Univariate Gaussian)
3. Parameter Estimation for Maximum Likelihood (Multivariate Gaussian)
4. Parameter Estimation for Maximum Likelihood (Gaussian Mixture Model)
5. Expectation Maximization Algorithm

IV. Neural Networks

Activation Functions

1. Activation Functions
2. Activation Function for a Bayesian Classifier

V. Supervised non-Parametric Estimation of Distributions and Classification

1. Overview
   • Kernel-based methods(Parzen Windows)
   • k-Nearest-Neighbor Estimation

VI. Linear Discriminant Functions or Single Layer Networks

1. Linear Discriminant Function
2. Single Layer Neural Network

VII Support Vector Machine

1. Optimization problem for an SVM dealing with Separable Dataset

VIII Neural Networks (CNNs)

1. Translational Invariance in CNNs
2. Pooling Functions
3. Neural Network Formulae for Calculations

IX Unsupervised Learning and Clustering

X Misc

• Exam Prep
• Important Concepts