Karteikarten
Cda
Karten in diesem Stapel (23)
- 1AAArchetypal Analysis
Dimension reduction
Freak representation
Extracting dominant convex hull
Similar to k-means but extremes/freaks
20% extreme mustache, 30% big nose - 2ICAIndependent Component Analysis
Dimension reduction, demixing
Maximize kurtosis, or non-gaussianity by imposing independence (rotation of PCA components)
Prewitening
Cocktail party problem - 3Question TextAnswer Text
- 4Sparse codingDimension reduction
Restrict the rotational issues for PCA, and put some L1 regularization on top, gives Gabor like features like how the brain stores information.
Unique - 5PCA & RotationsPCA does not give unique solutions.
If you rotate scores and loading you will get the same prediction performance using another solution.
Gives some tricky ness with interpretation.
Leveraged in ICA and sparse coding. - 6Question TextAnswer Text
- 7Question TextAnswer Text
- 8Question TextAnswer Text
- 9Question TextAnswer Text
- 10Ridge regressionRegression, linear
L2 regularization
P>>n problems are now ok (non-singular)
Shrinks coefficients simultaneously, but not to 0
handles collinearity better (grouped)
(XtX + lambda I)-1 Xty
Centering & Scaling
Increasing bias while reducing variance - 11LassoRegression, linear
L1 regularization
Shrinks to 0 to give sparsity while reducing variance.
LARS
Cyclical coordinate descent
Scaling and centering - 12Elastic NetPen Regression, linear
Combination of L1 and L2 norm
Control trade off between sparsity and shrinkage.
Fine grid for lambda (L2), more coarse for alpha(L1) - 13LDALinear Discriminant Analysis
Linear classification, assume normality & same correlation structure.
PCA like but tries to maximize separability between classes instead of max variance.
Probabilistic view.
Max (mean1 - mean2)/(s2-s2)
Both distance between means and min difference in std.
Max between class variance relative to the within class variance - 14KNNK-nearest neighbor
Classification, no assumptions
Flexible
Majority voting or average
Low K: high variance
High K: high bias
Assign value based on average for the k nearest neighbors. Distance metric e.g. Euclidean - 15Model selectionTrain test validation
Cross-validation
1 std error rule
AIC
BIC
Cp statistics - 16Cross-validation1 std error rule
Considerations:
Norm within fold
Impute separately in each fold
Consistent with groups
Time dependent data (corrs)
All preprocessing in each fold - 17Error gapsHuman level error
(Bias knob - more complex model, hyperparam)
Training error
(Regularization, more data, hyperparam)
Test error - 18Information criteriaEstimation of optimism for training errors
Based on inf theory (many obs)
AIC
BIC
Cp-statistics
Goodness of fit vs simplicity
BIC picks right if n=inf, AIC tends to pick too complex models. - 19AICGoodness of fit vs simplicity
Akaike Information Critera
Log-likelihood to estimate error
Same as cp-stats for Gaussian
AIC=err + d/N sigma2
D=effective params in model
Generally too complex model, asymptotically equivalent to leave one out CV. - 20BICBayes Information Criteria
Goodness of fit vs simplicity
Log likelihood loss function
Tends to pick too simple model for small n.
Picks right for n=inf
Only valid for n>>p
Larger penalty in BIC than AIC, but same concept. - 21BootstrappingAssess statistical accuracy
Randomly pull rows of data with replacement (so you can pick it again)
Run model with the same obs size but with random data (with some duplicates), do this a lot of times.
Out-of-bag estimates (like CV)
Stds 200-500 times
ConfIntervals 2000 times - 22Confusion MatrixClassifier performance as a table, 2x2
Test outcome vs true outcome
Type 1 (false alarms) vs Type 2 errors (misses) - 23ROC and AUCReceiver Operating Characteristics
By changing cut-off value you can get different true positive (sensitivity) vs false positive rates and plot them.
Area under the curve can be used for reporting of how good it became.
Informationen
- Beschreibung
- Karteikarten am Cda, erstellt von daniel Sahlin am 18/05/2019.
- Erstellt von
- daniel Sahlin
- Aufrufe
- 0
- Bewertung