PCA

PCA

1. Step 1: Covariance /Covariance Matrix of the dataset

   • What is covariance for two variable
   • Covariance for Multivariate
   • What is covariance matrix
   • Step 2:Calulate Eigenvalue/Eigenvector of covariance matrix of the dataset
     • What is eigenvalue & eigenvector
     • Why we calculate eigenvalue & eigenvector of the covariance matrix
       • later to rotate data alongside the axis of the eigenvectors: create new axis
       • it captures the variances of the data: first eigenvector is the direction of greatest variance of the dataset
         • each eigenvalue captures a propotion of ori variance
       • All matrix has a property of eigenstuff
     • Step3: Sort the eigenvalues by decresing order to rank the eigenvectors
       • Step 4: Select k eigenvectors which correspond to the k largest eigenvalues
         • k is the dim of new feature space
         • Step 5: Comput projection matrix; from the k selected eigenvectors, construct a projection matrix
           • Step 6: transform; tranform the original dataset with d dim with projection matrix W, obtain new k-dimension feature space
   • Why we use covariance matrix: because it captures the relationship of the data

2. Motivation for PCA

3. Step 0: Standarize dataset of dim(d)