Now you know some theories about Principal Components Analysis (PCA) and now we are going to go through how to actually perform it.

Next we would take a live dataset and actually perform PCA using R.

- 1. The First Principal Component (PC1)
- 2. The Second Principal Component(PC2)
- 3. About Scores and Loadings and Matrices
- 4. Interpreting Principal Components

As a brief introduction, keep in mind that the principal components are obtained by matrix multiplication. Precisely, we are would have to multiply the original data matrix X with loadings matrix Φ. This would give us the scores. So you can jump to About Scores and Loadings, and then come back to continue

**1. The Second Principal Component (Z2)**

We have out data X which is n x p dataset. Which means n observations (records or rows) and p features(or variable). This is represented as shown in X matrix.

We define the first principal components of a set of features X_{1}, X_{2}, . . . , X_{p} as the normalized linear combination of the features/variables in the original data:

Z_{1} = Φ_{11}X_{1} + Φ_{21}X_{2} + . . . + Φ_{p1}X_{p} ——- (PC1)

that has the highest variance.

The elements Φ are called the loadings of the first principal component and they are such that the sum of square of the Φs equal to 1 as in;

The combinations of the loadings make up the principal components loading vector (Φ_{11} Ф_{21} . . . Ф_{p1})^{T} and the combination of the loadings vector make up the loadings matrix.

X_{1}, X_{2}, . . . , X_{p} are normalized linear combination of the feature variables which means that they have a mean equal to zero and a standard deviation equal to one.

**2. The Second Principal Component (Z2)**

The second principal component Z2 is also a linear combination of the predictor variables (features) that has has the second-highest variance following Z1. It should also be noted the Z2 is uncorrelated with Z1 and that means that the direction of Z1 and Z2 are orthogonal as shown in Figure 1.0.

Z_{2} = Φ_{12}X_{1} + Φ_{22}X_{2} + . . . + Φ_{p2}X_{p} ——- (PC1)

Additional Components can be computed in a similar manner.

Once we are done computing the principal components say, (Z1 and Z2) we can then plot them on a plane one against the other on a 2d plot. The output is similar to Figure 1.0

**3. About Scores and Loadings**

In **PCA** we kind of break down the original data matrix X and break it down into item:

- set of eigen values,
**λ** - set of eigen vectors
**, Φ**

These two together make up the eigen pair. Now let’s picture what the set of eigen vector would look like. This is shown in the matrix below and this is called the **loadings matrix**.

Note the following about the loadings matrix

- this matrix is a square matrix ( has same rows and columns)
- the dimension of this matrix (
**p x p**) is comes from the number of features of the original matrix**X**

Each column of the loadings matrix corresponds to one principal component.

for the first column, we have the following elements (Ф_{11} Ф_{21} . . . Ф_{p1}). This is the loadings of the first principal component

for the second column, we have the following elements (Ф_{12} Ф_{22} . . . Ф_{p2}). This is the loadings of the second principal component

for the third column, we have the following elements (Ф_{13} Ф_{23} . . . Ф_{p3}). This is the loadings of the third principal component

for the **p**th column, we have the following elements (Ф_{1p} Ф_{2p} . . . Ф_{pp}).. This is the loadings of the **p**th principal components

Now let’s attempt to multiply the matrix **X** and **Ф** and let’s see what we would obtain. Let’s call this matrix** XФ**. This operation gives us a set of numbers called the **scores** (just a set of 1 dimensional values)

**X** is n x p matrix

**Ф** is p x p matrix

Therefore, the multiplying **X** by **Ф** would give us just a one dimensional data. If you are not sure of this you can go through a Brief Review of Matrix Operations made especially for this tutorial.

The product of the two matrices would give us the following elements:

Taking the first row of X and the first column of Ф, we would have:

x_{11}Ф_{11} + x_{12}Ф21 + . . . + x_{1p}Ф_{p1} (this is the first principal component (PC1)

Taking the second row of X and the second column of Ф, we would have:

x_{21}Ф_{12} + x_{22}Ф_{22} + . . . + x_{2p}Ф_{p2} (this is the second principal component (PC2)

And so on

**4. Interpreting Principal Components**

When we perform PCA on our n x p dataset, we obtain a loadings matrix Φ which is also an n x p matrix. We can then use just the first few columns(say 2 columns) of this matrix as our principal components and project our data along these directions.

When we perform PCA, we are projecting the data from p-dimensional space into two dimensional space

What does all this mean. Let’s illustration by using a particular dataset. This would help us to see how our dataset is affected after performing PCA on the data.

Now that you understand how to perform Principal Components Analysis, let’s now move a similar topic which is Singular Value Decomposition.

## One Comment on “PCA Tutorial 2 – How to Perform Principal Components Analysis (PCA)”