This is Lecture 6 of Machine Learning 101. We would discuss Polynomial Curve Fitting.

Now don’t bother if the name makes it appear tough. This is simply a follow up of Lecture 5, where we discussed Regression Line.

So as before, we have a set of inputs

**x** = {x_{1}, x_{2}, . . . , x_{n}}^{T} where N = 6

corresponding to a set of target variables:

**t** = {t_{1}, t_{2}, . . . , t_{N}}^{T} where N = 6

Our objective is to find a function that relates each of the input variables to each of the target values.

If we assume that the relationship is a linear one, then we can use the equation of a straight line given as:

y = ß_{0} + ß_{1}x

Then we simply calculates the coefficients ß_{0} and ß_{1}

However since we don’t know the nature of this relationship, we would extend this equation to cover more options.

So we would have it as:

which is the same as:

This is similar to what we already have. Just like y depends on *x* and *ß* in the linear model, also here, y depends on *x* and **w**.

M is the order of the polynomial. So if M is 1, then we have the linear model. If M is 2, then we have a quadratic function and so on.

**So what is w?**

**w** is simply the polynomial coefficients. So *w _{0}, w_{1}, . . . , w_{M}* are denoted by the vector

**w**.

So the problem reduces to simply determining the polynomial coefficients. Once we have it, we simple plug it into the polynomial and solve for *x.*

**How do we determine w?**

We determine w by fitting the polynomial to the training data set. This is achieved by minimizing the error function that measures the difference between the function y(x,w), for any given value of w and the corresponding point in the training data set.

To perform minimization, we need the error function. A good choice is to use the sum of squares error between the predicted values* y(x _{n}, w)* for each training data point and the corresponding target values

*t*

_{n}.This error function is given by:

where 1/2 is a factor added that would be explained later.

The value of this function would always be non-negative. It can also be zero, but rarely. It is zero if and only if the function produces exactly same output as the training set.

In summary, we need to choose the values of **w** that would produce the least value of E(**w**).

But another question is: **How to we choose M,** that is the order of the polynomial?

I would explain this in the next lecture 6

**Python Code for Polynomial Regression**

import numpy as np import pandas as pd import matplotlib.pyplot as plt from sklearn.linear_model import LinearRegression from sklearn.preprocessing import PolynomialFeatures # Read the dataset into a dataframe df = pd.read_excel(r'C:\ML101\Curve.xlsx') # Place the ranges in a variable and preprocess x = df['x'].values y = df['y'].values x = x.reshape(-1, 1) # Change the order here. degree is same as M poly = PolynomialFeatures(degree=11) # Fit a Polynomial Curve X_poly = poly.fit_transform(x) poly.fit(X_poly, y) linreg = LinearRegression() linreg.fit(X_poly, y) y_pred = linreg.predict(X_poly) # Plot the curves. The regression line is in red plt.scatter(x,y, color='blue') plt.plot(x, y_pred, color='red') plt.show()

## One thought on “Machine Learning 101 – Polynomial Curve Fitting”