This is the second lecture on classification. It follow the first one: Introduction to Classification.

**Bayes’ Classifier** is a classifier that works based on Bayes’ Theorem. It assigns each observation to the most likely class given the values of the measurements.

Also remember that Bayes’ theorem helps us compute conditional probabilities. So a Bayes’ classifier determines the conditional probability for the particular class given the features.

*Let’s take the X-ray example.*

When an X-ray image of the patient is obtained, the objective is to decide which of the two classes to assign it. Either C_{1} or C_{2}.

There for we need to compute the conditional probabilities of the classes C_{1} and C_{2} given the image. Let’s say C_{k}, where k=1, 2.

The conditional probabilities is given by:

In this equation:

P(C_{k}) is the prior probability for the class C_{k}. This is because the probability is know prior to taking the X-ray.

P(C_{k} | **x**) is the corresponding posterior probability. This is the probability of C_{k} after taking the X-ray. That is after **x** has been determined.

If the objective is to reduce the chance of assigning x to the wrong class, then we would choose the class that have the higher posterior probability.

In the case of two-class problem we just discussed, then Bayes’ classifier corresponds to trying to predict has P(C_{k} | **x**) > 0.5. Therefore the corresponding posterior probability would be < 0.5

Since we know that P(C_{1} | **x**) + P(C_{2} | **x**) = 1.

Bayes’ classifier is know to do pretty good job in classification. However, there are times when misclassification occurs.

We would end this lesson here so in the next lesson, we discuss how to minimize misclassification rate in Bayes’ classifier.

You could read up Basics of Conditional Probability

## 2 thoughts on “Machine Learning 101 – The Bayes’ Classfier”