These is a very important concept in Machine Learning and that is what we are going to cover today.

**Table of Content**

- What is Maximum Likelihood Estimation(MLE)?
- Properties of Likelihood Extimates
- Deriving the Likelihood Function
- Log Likelihood
- Applications of MLE
- Final Thoughts

**1. What is Maximum Likelihood Estimation?**

The likelihood of a given set of observations is the probability of obtaining that particular set of data, given chosen probability distribution model.

MLE is carried out by writing an expression known as the Likelihood function for a set of observations. This expression contains an unknown parameter, say, θ of he model. We obtain the value of this parameter that maximizes the likelihood of the observations. This value is called maximum likelihood estimate.

Think of MLE as opposite of probability. While probability function tries to determine the probability of the parameters for a given sample, likelihood tries to determine the probability of the samples given the parameter.

**2. Properties of Maximum Likelihood Estimates**

MLE has the very desirable properties especially for very large sample sizes some of which are:

likelihood function are very efficient in testing hypothesis about models and parameters

they become unbiased minimum variance estimator with increasing sample size

they have approximate normal distributions

**3. Deriving the Likelihood Function**

Assuming a random sample *x _{1}, x_{2}, x_{3}, … ,x_{n}* which have joint probability density and denoted by:

*L(θ)*=

*f(x*

_{1}, x_{2}, x_{3}, … ,x_{n}|*θ)*

*L(θ)*=

*f(x*

_{1}, x_{2}, x_{3}, … ,x_{n}|*θ)*

*f(x*

_{1}, x_{2}, x_{3}, … ,x_{n}|*θ)*as a product of univariates such that:

*L(θ)*=

*f(x*

_{1}, x_{2}, x_{3}, … ,x_{n}|*θ)*= f(

*x*

_{1}*|*θ) +f(

*x*

_{2}*|*θ),

*x*

_{3}*|*θ) +… +

*x*

_{n}*|*θ)

So the question is ‘what would be the maximum value of θ for the given observations? This can be found by maximizing this product using calculus methods, which is not covered in this lesson.

**4. Log Likelihood**

*maximum likelihood estimate*‘ for the given function

**5. Applications of Maximum Likelihood Estimation**

**6. Final Thoughts**