Data Mining

Data Mining

1. 2. Clustering

   • 2.1 What is Cluster Analysis
     • Types, Aspects, Cluster Analysis
   • 2.2. K-Means Clustering
     • Algorithm
     • Convergence Criterions
     • Evaluating K-Means Clusterings
     • Weeknesses: Inital Seed, Outliers
   • 2.3 Density-Based Clustering
     • DBSCAN
     • How to Determine suitable Eps and MinPts values
     • When DBSCAN Does Not Work Well
   • 2.4 Hierarchical Clustering
     • Strengths, Main Types, Algorithm
     • How to Determine Inter-Cluster Similarites
       • Single vs. Complete Link
       • Group Average
     • Problems and Limitations
   • 2.5 Proximity Measures
     • 2.5.1 Proximity of Single Attributes
       • Levenshtein
     • 2.5.2 Proximity of Multidimensional Data Points
       • Similarity of Binary Attributes
         • Symmetric Binary Values -> SMC
         • Asymmetric Binary Attributes -> Jaccard
         • Using Weights to combine Similarities
       • for numeric Values
         • Euclidean Distance
           • Normalization
   • How to choose a good Clustering Algorithm

2. 1. Introduction

   • 1.1 What is Data Mining?
   • 1.2 Methods and Applications
     • 1.2.1 Clustering
     • 1.2.2 Classification
     • 1.2.3 Regression
     • 1.2.4 Association Rule Discovery
     • 1.2.5 Time Series Prediction
     • 1.2.6 Sequential Pattern Discovery
     • 1.2.7 Anomaly Detection
   • 1.3 Data Mining Process
     • 1.3.1 Selection and Exploration
     • 1.3.2 Preprocessing and Transformation
     • 1.3.3 Data Mining
     • 1.3.4 Deployment

3. 3. Classification

   • 3.2 K-Nearest Neigbor
     • Basic Idea & Requirements, Algorithm
     • Pro und Con
     • Lazy Learning vs. Eager Learning
   • 3.1 What is Classification
   • 3.3 Decision Tree Classifier
     • Definition, Decision Boundary
     • Decision Tree Induction
       • Hunt's Algorithm
       • 3.3.2 Determine how to split the record
         • Determine when to stop the split
         • How to determine the best split
           • 3.3.2.1 GINI
             • for a given node t
             • node split
               • Binary Attributes
               • Categorical Attributes
             • Continuous Attributes
           • 3.3.2.2 Information Gain & Entropy
           • 3.3.2.3 GainRATIO
           • 3.3.2.4 Classification Error
       • 3.3.1 Test condition
         • attribute types
           • nominal, ordinal, continous
         • way of split
           • 2-way, multi-way Split
       • 3.3.3 Overfitting
         • Pre-Pruning
           • Stop the Algorithm before tree becomes complete
         • Post-Pruning
           • Trim the nodes bottom up after the tree is fully developed
           • Estimate the Generalization Error
         • Occam's Razor
       • Advantages and Disadvantages
   • 3.4. Model Evaluation
     • 3.4.1 Metrics for Model Evaluation
       • Confusion Matrix
         • Accuracy & Error Rate
         • Class Imbalance Problem
         • Precision and Recall
       • Cost Sensitive Model Evaluation
       • ROC Curves
     • 3.4.2 Methods for Model Evaluation
       • Holdout Method
       • Random Subsampling
       • Cross Validation (K-Fold)
       • Add: Increase model performance
   • 3.5 Rule Learning
     • 3.5.1 Applying a Rule-Based Classifier
       • Rule Coverage & Accuracy
       • Mutually Exclusive, Exhaustive
     • 3.5.2 Learning Rule-Based Classifier
       • 3.5.2.1 Indirect Method
         • C4.5
       • 3.5.2.2 Direct Method
         • For 2-class problems
         • For multi-class problems
         • RIPPER - Sequential Covering
   • 3.6 Naive Bayes
     • Bayes Theorem and Applying
     • Prior and Class-Conditional Probability
     • Handling Numerical Attributes
     • Handling Missing Values
     • Zero-Frequency-Problem
     • Characteristics
     • Alternative Classification Methods
   • 3.7 Artifical Neural Networks
     • What is ANN? DNN?
     • Algorithm for Training ANN?
     • Characteristics
   • 3.8 Support Vector Machine
     • SVM and Hyperplane
     • Dealing with Non-Linearly Separable Data
     • Characteristics
     • Tuning SVM
   • 3.9 Parameter Tuning
     • Parameter Optimization
     • Attribute Subset Selection
     • Outputing Interim Results

4. 4. Regression

   • 4.1 What is Regression
     • Regression Model Learning
     • Regression Techniques
   • 4.2 KNN for Regression
   • 4.3 Model evaluation
     • 4.3.1 Methods for Model Evaluation
     • 4.3.2 Metrics for Model Evaluation
   • 4.4 Regression Trees
     • Difference to DT
     • Fitting
   • 4.5 ANNs for Regression
     • Defintion
     • Least-Squares Approach and Ridge Regression
   • 4.6 Linear Regression
     • Assumption
     • Least-Suqare Approach & Ridge Regularization
     • Feature Selection
     • Interpolation vs. Extrapolation
     • Linear Regression vs. k-NN Regression
     • Non-Linear Problems
   • 4.7 Polynomial Regression
   • 4.8 Local Regression
     • Advantages & Disadvantages
   • 4.9 Time Series Forecasting
     • Data-driven: Smoothing
       • SMA
       • EMA
     • Model-driven
       • Component Models
       • Windowing

5. 5. Text Mining

   • 5.1 What is Text Mining?
     • Sentiment Analysis
     • Search Log Mining
     • Search vs. Discovery
     • Information Extraction
   • 5.2 Text Preprocessing
     • Tokenization
     • POS Tagging
     • Stopword Removal
     • Stemming
   • 5.3. Feature Creation
     • Bag of Words
     • term-frequency, TD-IDF
   • 5.4. Feature Selection
     • Pruning Document Vector
     • Filter Tokens
   • 5.5 Pattern Discovery
     • 5.5.1 Document Clustering
       • Jaccard Coefficient
       • Cosine Similarity
     • 5.5.2 Document Classification
       • Example: Sentiment Ana
       • Tricks

6. 6. Association Analysis

   • 6.1 Correlation Analysis
   • 6.2 Association Analysis
     • 6.2.1 Frequent Itemset Generation
       • Brute Force Approach
       • Apriori Principle -> Pruning
     • 6.2.2 Rule Generation
     • 6.2.3 Handling Continuous and Categorical Attributes
     • 6.2.4 Interestingness
       • Drawback of Confidence
       • Lift
     • Definition
     • Support and Frequent Itemset
     • Association Rule Mining

7. Generalization Error

   • Bias
   • Variance
   • Irreducible Error

8. Model Selection, Parameter Tuning, Learning Curve