Data Science

Data Science

1. What is it?

   • Data Science is a multidisciplinary field that uses techniques from statistics, machine learning, and data analysis to extract valuable insights and knowledge from data, enabling data-driven decision-making and solving complex problems.
     • Data Science is applied across various sectors, including finance, healthcare, marketing, and more, to solve problems, make predictions, and guide strategic decisions based on data-driven evidence. It's a dynamic and evolving field at the intersection of computer science, statistics, and domain expertise.

2. What is data analytics?

   • Data analytics is the process of examining, cleaning, transforming, and interpreting data to discover meaningful patterns, insights, and information that can be used to support decision-making, improve business processes, or gain a deeper understanding of a particular phenomenon. It involves the application of statistical, mathematical, and computational techniques to data in order to draw conclusions and make informed recommendations.
     • Key aspects:
       • Exploratory Data Analysis
       • Descrpitive Analysis
       • Diagnostic Analysis
       • Predictive Analytics
       • Prescriptive Analytics
       • Statistical Analysis
       • Data Collection, Cleaning and Preprocessing

3. What is big data?

   • Big data refers to extremely large and complex datasets that are beyond the capacity of traditional data processing and management tools to handle efficiently. These datasets are characterized by the three "V"s:
     • Volume
       • Big data involves a vast amount of data, typically ranging from terabytes to petabytes or even exabytes. This data can come from various sources, including sensors, social media, e-commerce transactions, and more
     • Velocity
       • Big data is generated and collected at a high velocity or speed. Data streams in real-time or near-real-time, making it challenging to process and analyze using traditional batch processing methods.
     • Variety
       • Big data is diverse and comes in various formats, including structured data (e.g., databases), semi-structured data (e.g., XML, JSON), and unstructured data (e.g., text, images, videos). Analyzing and extracting insights from this heterogeneous mix of data types can be complex.
     • To effectively handle big data, organizations often turn to specialized tools and technologies, such as distributed computing frameworks like Hadoop and Spark, NoSQL databases, cloud computing services, and data warehousing solutions. These technologies enable the storage, processing, and analysis of large datasets in a scalable and cost-effective manner. Big data analytics involves applying advanced analytics techniques, including machine learning, data mining, and deep learning, to extract valuable insights and patterns from big data. This can help organizations optimize operations, improve customer experiences, detect fraud, make predictions, and more.

4. How does Data Science relate to both films?

   • Margin Call
     • The excerpt shown of the movie Margin Call is related to Data Science because as it was shown in the video, apparently a member of the meeting told the boss of the company that certain data that was analyzed projected huge losses for the company. This is basically what science data does and it is why it is such a powerful tool. With certain data and if analyzed the correct way, we can project and anticipate to future situations that may produce problems for a company, even if everything seems fine, certain patterns may lead to unfortunate situations, and with data science we are able to identify and prevent them.
   • The Big Short
     • The movie of The Big Short is a perfect example of how powerful data is, and what advantages it may bring to you if analyzed correctly. In the case of the movie, the main character finds some irregularity with mortgages by analyzing data by himself. He then realizes he can take advantage of the situation and talks to his boss. The lesson here is that data can give us a lot of information, it may look like a lot of rows and columns filled with numbers, but I we are able to analyze them and find accurate patterns, then everything turns predictable, and there is where people take advantage.

5. CRISP DM Methodology

   • The CRISP-DM (Cross-Industry Standard Process for Data Mining) methodology is a widely accepted framework for carrying out data mining and data analytics projects. It provides a structured approach to guide the various stages of a data mining project from start to finish. CRISP-DM consists of six main phases:
     • 1. Business Understanding:
       • In this initial phase, the project's objectives and requirements are defined from a business perspective. Key questions to address include: What are the business goals? What does success look like? How will the project impact the organization?
     • 3. Data Preparation:
       • Data preparation is a crucial phase where data is cleaned, transformed, and formatted for analysis. Tasks include data cleaning (handling missing values, outliers), feature engineering (creating new variables), and data integration (combining data from various sources).
     • 2. Data Understanding:
       • Data understanding involves exploring and familiarizing oneself with the data available for the project. Activities include data collection, data description, data quality assessment, and the identification of potential data issues or challenges.
     • 4. Modeling:
       • The modeling phase focuses on selecting appropriate modeling techniques (e.g., regression, clustering, classification) and building predictive or descriptive models using the prepared data. Model parameters are selected and fine-tuned, and model performance is evaluated using various metrics.
     • 5. Evaluation:
       • In this phase, the quality and effectiveness of the models are assessed based on predefined criteria and business objectives. Common evaluation techniques include cross-validation, hold-out validation, and the use of performance metrics like accuracy, precision, recall, and F1-score.
     • 6. Deployment:
       • Once a satisfactory model is developed and evaluated, it is deployed into a production environment or integrated into the business process to make data-driven decisions. Deployment may involve creating dashboards, reports, or APIs to deliver results to end-users.