Copy and Edit

Designing and Developing the ETL System

Description

Mind Map on Designing and Developing the ETL System, created by Thean Olivier on 22/04/2018.
Thean Olivier
Mind Map by Thean Olivier, updated more than 1 year ago
Thean Olivier
Created by Thean Olivier over 7 years ago
4
1
0

Resource summary

Designing and Developing the ETL System
  1. Step 1 — Draw the High Level Plan
    1. Step 2 — Choose an ETL Tool
      1. Self-documentation that comes from using a graphical tool.
        1. ETL tools offer advanced transformation logic.
          1. Improved system performance at a lower level of expertise.
          2. Step 3 — Develop Default Strategies
            1. Step 4 — Drill Down by Target Table
              1. Ensure Clean Hierarchies
                1. Develop Detailed Table Schematics
                  1. Develop the ETL Specification Document
                    1. Develop a Sandbox Source System
                      1. Develop a Sandbox Source System
                      2. Step 5 — Populate Dimension Tables with Historic Data
                        1. Populate Type 1 Dimension Tables
                          1. Dimension Transformations
                            1. Dimension Table Loading
                            2. Populate Type 2 Dimension Tables History
                              1. Populate Date and Other Static Dimensions
                            3. Step 6 — Perform the Fact Table Historic Load
                              1. Historic Fact Table Extracts
                                1. Fact Table Transformations
                                  1. Fact Table Loading
                                    1. Test, Test, and Test Again
                                      1. Develop Incremental ETL Processing
                                      2. Step 7 — Dimension Table Incremental Processing
                                        1. Identify New and Changed Dimension Rows
                                          1. Process Changes to Dimension Attributes
                                          2. Step 8 — Fact Table Incremental Processing
                                            1. Fact Table Extract and Data Quality Checkpoint
                                              1. Fact Table Transformations and Surrogate Key Pipeline
                                                1. Late Arriving Facts and the Surrogate Key Pipeline
                                                  1. Incremental Fact Table Load
                                                    1. Speed Up the Load Cycle
                                                    2. Step 9 — Aggregate Table and OLAP Loads
                                                      1. Step 10 — ETL System Operation and Automation
                                                        1. Schedule Jobs
                                                          1. Handle Predictable Exceptions and Errors Automatically
                                                            1. Handle Unpredictable Errors Gracefully
                                                              1. Maintain Database Objects
                                                                1. Develop and Test ETL Automation

                                                                Media attachments

                                                                Show full summary Hide full summary

                                                                Want to create your own Mind Maps for free with GoConqr? Learn more.

                                                                Similar

                                                                Browse Library