Data Science

Data Science

1. It is an interdisciplinary field that uses machine learning to obtain insights from big data and identify new problems.

   • Applications
     • Healthcare
       • Used for identifying cancer tumors, personalizing treatment plans and cleaning clinical trial data.
     • Transportation
       • Used for modeling traffic patterns, optimizing food delivery and improving package delivery.
     • E-commerce
       • Used for creating targeted ads, curating vacation rentals and predicting consumers' product interests.
   • Skills
     • Computer science
     • Programming languages
     • Data modeling and visualization
     • Statistics
   • Examples
     • In The Margin Call, Peter Sullivan concluded that debt from over-leveraged assets would bankrupt the company he was working for.
     • In The Big Short, Michael Burry discovered the US housing market was collapsing.

2. Data analytics

   • It is the science of analyzing existing information to find specific results and immediate solutions.
     • Applications
       • Web Search
         • Used for presenting best results of a search engine and spreading advertisements.
       • Military
         • Used for creating augmented reality simulations to help armies plan their next moves.
       • Manufacturing
         • Used for predicting the amount of products in demand for better operating capacity.
     • Skills
       • Data warehousing and analytics
       • Predictive modeling
       • Data mining
       • Data visualization tools
     • Examples
       • In The Margin Call, Peter Sullivan explains his analysis of the model to his boss, who decides to sell all his assets at once.
       • In The Big Short, in response to his findings, Burry decides to bet against mortgage-backed securities.

3. Statistical analysis

   • It is the process of collecting and analyzing data in order to discern patterns and trends.
     • Examples
       • In The Margin Call, we see Sullivan checking every over-leveraged asset that predicted the fall of his company.
       • In The Big Short, Burry is analyzing an Excel file with the purchase of every mortgage-backed security.
   • Big Data
     • It is data that contains greater variety and is presented in increasing volumes and at greater velocity.
       • Examples:
         • For both films, the big data are all the registered mortgage-backed securities related to the companies.

4. CRISP-DM (Cross-industry process for data mining)

   • It is an open standard model that describes the traditional approaches used by experts in data mining.
     • Phase 1: Business understanding
       • Defines the problem we want to solve.
         • Example
           • A used car dealer wants to estimate the price of a used car based on its attributes.
     • Phase 2: Data understanding
       • Gathers the data that will be used for the solution.
         • Example
           • Horsepower and length of the car are taken as attributes.
     • Phase 3: Data preparation
       • The data is arranged for the model.
         • Example
           • Every variable is placed as a column for a table.
     • Phase 4: Modeling
       • Creates a model that describes the data.
         • Example
           • An algorithm obtains the average price of the cars.
     • Phase 5: Evaluation
       • Validates the performance of the model.
         • Example
           • A variance test is set for horsepower and length.
     • Phase 6: Deployment
       • The model is finally published.
         • Example
           • The car price prediction model has reasonable accuracy.