Ohne Titel

Seite 1

Data Engineer

RDBMS tech (used for transactional processing)

Oracle

MySQL

Postgres

MS SQL Server

Sybase

Datawarehouse Tech (used for analytical processing)

Teradata

Databricks

Snowflake

GCP big query

AWS redshift

Azure Synapse

Purpose built database - for a given purpose, we will use a specific database technology and build our application.

Transactional purpose like payment, banking - RDBMS

Analytical - Datawarehouse or MPP (Massively parallel processing)

search based purpose like linkedIN search - Elastic search, SOIR

NoSQL (store images,audio,text,likes) - MongoDB, Cassandra

Graph - Neo4j

Typically, we use data lake or data warehouse for downstream data processing.

datalake - cloud storage like AWS S3, GCP GCS

data warehouse and data leak work on top of each other to provide some insights, visualization to end customers (using reports and dashboards)

Problems/ use cases for RDBMS - Buy a product from the retail company, Track the status of order, Cancel the order and get refund.

Problem/ use cases for data warehouse - What is the profitability of the organization, what is the revenue trend over this year, Which category is performing well?