Data Compression, Metadata, and Data-Driven Solutions

Data Compression, Metadata, and Data-Driven Solutions

1. Data Compression

   • Sometimes, we need to 'squeeze' files so they take up less space. But we need to do it smartly. Do we want to lose quality, or keep it 100% the same? That’s the key difference between lossless and lossy compression.
     • Lossless Compression
       • No data is lost. When decompressed, you get the exact original file. Examples are ZIP files, PNG images, GIFs. It looks for repeating patterns (like "111111" turns into "6x1"). We use lossless compression when data accuracy matters (text files, software files).
     • Lossy Compression
       • Some data is permanently removed to make the file smaller. Examples are MP3 audio, JPEG images, YouTube videos. It removes "unnecessary" details (like frequencies the human ear can’t hear). We use lossy compression when space matters more than perfection (music, images, video).
     • How to Choose Between Lossless vs. Lossy
       • Text/Code/Software? → Lossless (You can’t afford to lose accuracy here.) Music/Video/Images? → Lossy (Small quality loss is okay for better file sizes.)
   • Exam Tip: "If the exam asks you which compression method is better, it depends on the file type. For text files, it’s always lossless. For music, images, or videos, it’s lossy."

2. Compression Algorithms

   • Run length Encoding
     • Replaces long runs of identical data with a short description. Example: "AAAAAA" becomes "6A". Best for data with lots of repetition (like simple images or long sequences of the same number).
   • Huffman Coding
     • Uses shorter binary codes for more frequent characters. Example: If "e" is common, it gets a short code, while rare letters like "z" get a longer code. Works best for compressing text.
   • How do algorithms impact file size?
     • Some files (like random data) don’t compress well. Files with lots of repetition (like pixel art) compress really well.
   • Exam Tip: "You might be asked to apply Run-Length Encoding or Huffman Coding to a sequence of data. Be ready to trace through it and calculate the compressed size."

3. Metadata

   • What is Metadata?
     • Data that describes other data. Examples: File name, file type, and date modified for a file. GPS location for a photo. Timestamps for a social media post.
   • How metadata helps?
     • Sorting: "Sort files by date, name, or size — that’s metadata at work." Filtering: "Want to find all files from December 2024? Metadata makes it easy." Categorizing: "Tagging posts on social media lets you filter by topic."
   • Why Metadata Matters on the AP Exam
     • You may be asked to analyze a dataset and look for specific metadata (like time or location). Example: "What can be learned from the metadata of this dataset of photos?" Possible answers: time of day, location, or device used.
   • Exam Tip: "If you see words like timestamp, location, file type, or tags in the question, you’re dealing with metadata. Don’t overthink it.. it’s just 'data about data'."

4. Data driven solutions

   • What is a data driven solution?
     • A program or process that collects, analyzes, and uses data to make decisions. Example: A GPS app that recommends the fastest route uses data from other drivers.
   • How to solve a data problem
     • Step 1: Identify the Problem — "What’s the goal? Are we looking for patterns or predictions?" Step 2: Collect Data — Gather information from users, sensors, or APIs. Step 3: Process Data — Clean it, filter it, and get it ready for analysis. Step 4: Analyze Data — Look for patterns or trends. Step 5: Present the Results — Display charts, graphs, or dashboards.
   • Tools used for data analysis
     • Spreadsheets (like Excel or Google Sheets). Coding (Python, R, or SQL to manipulate and analyze large datasets). Data Visualization (like bar graphs, pie charts, and scatter plots).
   • Exam Tip: "If you see a problem where you have to suggest a 'process to analyze data,' use the 5-step method: Identify, Collect, Process, Analyze, and Present."

5. Real world scenarios

   • Scenario 1: Streaming Services (Netflix, Spotify, YouTube)
     • They use lossy compression for videos and music to save storage space. Metadata tracks what you watch, when, and for how long (to recommend new shows). They analyze your watch history and make personalized recommendations.
   • Scenario 2: Social Media (Instagram, TikTok)
     • Images and videos are compressed to load quickly. Posts have metadata like location, tags, and timestamps. Algorithms use engagement data (likes, comments) to decide what content to show you next.
   • Scenario 3: Fitness Trackers (Fitbit, Apple Watch)
     • Sensor data (like heart rate) is compressed to save battery. Tracks location, time, and workout type. Tracks your progress over time and gives you fitness goals.
   • Exam Tip: "If you see a question about how a business uses data, remember that they collect, process, and analyze it to improve products or give recommendations."