GoConqr has a new look! Let us know what you think.

Sign up free

Flashcards

FINC781

toby riley English 49 cards

Cards in this deck (49)

  1. 1What are the main steps of ML?1. Get the “Big picture” (what problem are you trying to solve – and why?)
    2. Data (what data do you have – and what data should you get?)
    3. Look at the data (understand the structure, relationships, miss- ing data)
    4. Prepare the data (cleaned up, in table form)
    5. Select a model and train it (but which model? No free lunch theorem)
    6. Fine-tune the model (hyper-parameters, performance)
    7. Present the solution (to whoever paid for it)
    8. Put solution into production
  2. 2What is ML?Machine Learning is the science (and art) of programming computers so they can learn from data.
  3. 3Supervised vs Unsupervised?Supervised are given labels while unsupervised are unlabelled, and thus semi-supervised are when part of the data is labelled and is used to help assign labels to unlabelled instances
  4. 4Batch vs OnlineBatch is when model is trained on the entire training data while online means the model can be trained incrementally where it is fed data sequentially
  5. 5Instance based vs model based?Instance based is where system learns instances by heart and applies a similarity matrix to apply to new instances, while model based is when a model is built that generalises from a set of examples and uses it to make predicitons.
  6. 6Challenges of ML?Insufficient data, representative training data, irrelevant features, overfitting or underfitting,
  7. 7What is feature scaling and how to deal with missing values?ML algorithms work better if input numerical attributes have different scale, can fill with median or mean of training data
  8. 8What is K-fold cross validation?Splitting the training data up into K partitions and then trains and evaluates the model, picking a different fold for for evaluations and training on the K-1 remaining folds.
  9. 9The relationship between threshold precision and recall?Recall will allways go down when threshold is increased while precision can go either way (usually up).
  10. 10What does the ROC plot? and when to use PR curve instead?True positive rate (recall) against false positive rate.
    you should prefer the PR curve whenever the positive class is rare or when you care more about the false positives than the false negatives.
  11. 11How to compare the ROC of different models?Compare the area under the curve, the one with the largest is best. Using ROC AUC does not depend on a particular threshold like P and R do.
  12. 12Multi-class, multi-label and multi-outputMulti-class is when an instance can be classified as more than one option, multi-label is when there are multiple instances being classified at the same time but binary, while multi-output is multi-label and multiclass
  13. 13A closed form equation like the normal does what?directly computes the model parame‐ ters that best fit the model to the training set (i.e., the model parameters that minimize the cost function over the training set).
    Should only be used if you want a linear model and the features are not huge.
  14. 14What is general GD?Gradually tweaks the model parameters to minimize the cost function over the training set, eventually converging to the same set of parameters as the first method.
  15. 15Difference between bias and variance?Bias = large errors on training set (in-sample). Model does
    not perform.
    ◦ Variance = large errors on validation set (out-of-sample).
    Model does not generalise.
  16. 16What is the learning rate in GD?It is the size of the steps, ie how much you change the gradient of the paramater vector which decreases the MSE. Too small and the model will take to long to converge, too big and you make might the model diverge.
  17. 17Difference between stochastic, batch and mini batch?Batch calculates the gradient using the whole training set, stochastic uses a random training example and mini batch uses a sub sample.
    Stochastic is much faster but much more random, which can be better if there is a local minima but bad because it wont settle at the optimum. Mini batch is halfway between both.
  18. 18What does increasing a models complexity commonly do?Increases its variance and chances to overfit test data but reduces its bias, vice versa for reducing complexity.
  19. 19Ridge vs lasoo vs elastic netRidge forces the learning algorithm to not only fit the data but also keep the model
    weights as small as possible. Ridge adds the squared values of the errors.
    It adds the absolute values of the errors.
    lasoo eliminates weights of least important features
    Elastic net is combination of both
  20. 20When should you use ridge , lasso elastic net?If you think that not all features are important then use lasoo or elastic net, elastic net is usually preferred to lasoo as it is less erratic especially when the number of features is greater than the number of instances or when features are strongly correlated.
  21. 21What is early stopping?A technique with GD is putting a large number of iterations in the model then stopping the model when it reaches a validation error minimum, so as to not overfit the data
  22. 22What are good about decision trees?They are robust to uninformative features, to highly correlated features, and to linear scaling of features.
  23. 23What are the decision boundaries of a decision tree?Allways orthogonal, either vertical or horizontal straight lines.
  24. 24What does gini coefficient measure?Measures the impurity of a node, if every instance of a node belongs to the same class then it is pure and has a GC of 0.
  25. 25What does the CART algorithm do?The learning algorithm must select the combination of 1) feature and 2) cut-off point for that feature that splits the sub-sample at that point of the tree so as to best separate the classes in that sub-sample.
    – It does so by choosing a feature and feature cut-point that minimises the Gini impurity of the resulting two sub-branches, weighted by the observations in each sub-branch
  26. 26What will an unconstrained decision tree do?Will perfectly fit the data, essentially memorises it and overfits.
  27. 27What are the usual types of hyper parameters?Exact strucure of model - depth of tree, how many layers, order of polynomial
    learning algorithm - LR in GD, batch size, entropy or gini
    Data prep steps - feature scaling, dimensionality reduction,
  28. 28What are the important points on data sets?Whenever you use a dataset to search for good parameters or good hyper-parameters, that dataset can no longer be used to measure the performance of the model - it has been tainted by the search.
    Remember, you can use a dataset to either search for a good model, or to test a particular model, but never both
  29. 29Can a feature be used to split multiple nodes?Yes they can be, this is why it can model non-linearities .
  30. 30What is bagging and pasting?A method of splitting up data for different predictors (of the same type) to be trained on, bagging is performed with replacement so classifiers can share instances while pasting, they cannot. The combine predictors using hard or soft voting.
  31. 31Difference between hard and soft voting?Hard voting tallies up the predictions made by each classifier and pick the class with the most votes, soft voting aggregates the probabilities across the classifiers and selects the class with highest probability
  32. 32What are other methods of creating diversity asides from bagging and pasting?Training N classifiers on random subsets of features, Patches - on random features and instances, random forrests, boosting - ada and gradient.
  33. 33What are random forrests and extra trees?RF are a collection of decision trees trained using bagging - they are great for measuring the importance of each feature, extra trees is when random thresholds each feature are used instead of searching for the best ones.
  34. 34What is boosting - ada and gradient?boosting is the sequential adding of predictors trained to correct the errors by the previous. Ada - assigns higher weights to miss-classified instances
    Gradient boosting - next model is trained on the residual errors of the last - and they are trained using gradient descent (residuals only true if regression and RMSE) if trees its trained on the gradient on existing set of trees.
  35. 35What is the drawback to boosting vs other ensemble methods?Cant be done in parrallel and so does not scale as well as bagging or pasting
  36. 36In an ensemble with lots of predictors trained on smaller subsets of data what is the most likley outcome?Combining lots of models with slightly higher bias usually aggreats the bias to be about the same if it just a single model but the variance is usually better.
  37. 37What is stacking?Using a ML algorith to optimally combine the output of your N learners instead of voting or averaging.
  38. 38What is the curse of dimensionality?high-dimensional datasets are at risk of being very sparse: most training instances are likely to be far away from each other. This also means that a new instance will likely be far away from any training instance, making predictions much less reliable than in lower dimensions, since they will be based on much larger extrap‐ olations. In short, the more dimensions the training set has, the greater the risk of overfitting it. High features does lead to high dimension of targets.
  39. 39What to remember with PCA's when selecting the PC?That they must be orthogonol or uncorrelated with eachother.
  40. 40What is PCA? and how is it different to PLS?Projection onto a lower-dimensional hyper-plane, such that the total remaining variance in the lower dimension is as high as possible.
    PLS aims to reduce dimensions to try explain more variation while PCA hopes to retain. PLS is better for predicting something - supervised, while PCA better for unsupervised
  41. 41When dramtically reducing the number of features what problems can occur?New features are harder to interpret than older ones, new features will not retain as much information.
  42. 42After having lowered dimensions down to K as pre-processing what likley effect would having K+1 features have?Would have less of a regularising effect
  43. 43What are negatives of K-means clustering?K-Means does not behave very well when the clusters have varying sizes, different densities, or nonspherical shapes. You also have to select the number of clusters which might be a hassle.
  44. 44How does K-means work?You assign K number of instances (can be randomly) as centroids then assign labels based on the distance to centroids, then of those clusters take the centroid and so on until the centroids stop moving.
  45. 45What point should you pick when plotting inertia against K?Pick the elbow of the curve, as increasing K allways decreases inertia as instances are closer to centroids inertia is not allways the best metric to use.
  46. 46What is the difference between RMSE and and MAE?MAE is the difference between actual and predicted and does not conisder direction, while RMSE give higher weighting to bigger errors
  47. 47What is the manifold assumption, and its implicit assumption?Most real-world high-dimensional datasets lie close to a much lower-dimensional manifold. This assumption implicitly assumes that the task at hand (e.g., classification or regression) will be simpler if expressed in the lower-dimensional space of the manifold.
  48. 48How are neural networks trained?Back propogation
  49. 49What is the key difference between social scientists and machine learning?SS formulate hypothesis and then test those hypothesis with data (t-test) while ML doesn't care about the underlying theory it just makes predictions, and thus the proof is if it works it works. In Society, we care much more about results and thus ML has gained much more importance.

Information

Description
Flashcards on FINC781, created by toby riley on 24/05/2021.
Created by
toby riley
Views
0
Rating

Feedback

What is it about?

What do you think of the new look?

How do you like it?