Bayes Risk - Action and Conditional Risk

Conditional Risk

The Risk associated with taking a decision $a$ given the observation $\mathbf{x}$ or simply the cost of a decision is given by,

$$\boxed{R(a=j\mid \mathbf{x})=\sum _{i\in \mathcal{S}}\stackrel{cost}{\overbrace{\lambda (a=j\mid s=i)}}\underset{Posteriori}{\underbrace{P(s=i\mid \mathbf{x})}}}$$

where,

• $a\in \mathcal{A}=\{1,2,\dots ,A\}$ represents the action
• $\lambda$ is the loss function, such that $\lambda (a=j\mid s=i)$ states how costly an action $a=j$ given a classification $s=i$

Note: The Cost function is entered manually. So that the decision that would lead to a negative outcome is weighted with a very high cost.

Example

Consider a Two Class problem such that

1. Actions $a\in A=\{\text{Treat},\text{Don’t Treat}\}$
2. Classification $s\in S=\{\text{Sick},\text{Healthy}\}$ Loss function

• $\lambda (a=1\mid s=1)$ : Low
• $\lambda (a=1\mid s=2)$ : High
• $\lambda (a=2\mid s=1)$ : High
• $\lambda (a=2\mid s=2)$ : Low

Give that we use a Bayes Classifier to make diagnosis a on the Patient. The Conditional Risk can be calculated as follows.

$$\begin{array}{r}R(1\mid \mathbf{x})=\lambda (1\mid s=1)\cdot P(s=1\mid \mathbf{x})+\lambda (1\mid s=2)\cdot P(s=2\mid \mathbf{x})\\ R(2\mid \mathbf{x})=\lambda (2\mid s=1)\cdot P(s=1\mid \mathbf{x})+\lambda (2\mid s=2)\cdot P(s=2\mid \mathbf{x})\end{array}$$

Bayes Decision Rule for Actions

$$\boxed{a^{\ast }=\arg \underset{j\in \mathcal{A}}{\min }R(a=j\mid \mathbf{x})}$$

Thus the best action would be the one that minimizes the risk.