DWPrep

DWPrep

1. Data Quality & Privacy

   • APP
     • Part 1 - Privacy consideratioan for Personal Info
       • Principle 1 - Management of personal Info
         • What for? - The entities handle personal info in an Open and Transparent manner.  Entities must -  - Comply APP + deal with inquiries & complaints - Have up to date policy - Handle inquiries free of charge - Provide the response in an appropriate form - Able to provide Copy of the policy
         • (a) What type of information collected (b) How the info collected (c) the purposes of collection, holds, usage, and disclosure(d) How an individual access and corrects info? (e) How an individual can complain and it handled by entity (f) Is info released to overseas (g) If release to overseas, which countris
       • Principle 2 - Anonymity & Pseudonimity
         • Does Not Apply IF -- Identity required by Aus law, court or if the person identified themselves If it is not possible to serve individuals without the identity . e.g. a patient at hospital 
         • Individual have an otpion not to identify themselves
     • Part 2 - Collection of Peronal Info
       • Principle 3 - Collection of Solicited Personal Info
         • Who can collect? Agencies & Orgs MUST not collect unless directly related to its functions. MUST not collect info unless individual consents. EXCEPTIONS for law and order enforcements, Not for PROFIT for related functions o individuals directly related. E.g. Mental HealthOrgs. MEANS OF COLLECTION: MUST COLLECT DIRECTLY FROM THE INDIVIDUAL, UNLESS - IF THE INDIVIDUAL CONSENTS OR REQUIRED BY LAW OR IT IS IMPARACTICAL LIKE AN ACCIDENT/TRAUMA
       • PRINCIPLE 4 - UNSOLICITED INFO - RECEIVED FROM 3RD PARTY
         • IMMEDIATELY CHECK WHETHER IT COMPLIES WITH PRINCIPLE 3 - IF NOT SHOULD BE DESTROYED LAWFULLY. IF COMPLIES PRINCIPLES 5-13 SHOULD APPLY
       • PRINCIPLE 5 - NOTIFY ABOUT INFO
         • Notify the identity of APP Entitiy. How inforamtion collected. What are the consequences. How to access the info. How to complain. How to remore access. Who else has access.
     • Part 3 - Dealing with Personal Info
       • PRINCIPLE 6 - USE OR DISCLOSURE ABOUT PERSONAL INFO
         • ENTITY NOT ALLOW TO DISCLOSE FOR ANY OTHER PUSPOSE THAN THE COLLECITON PURPOSE UNLESS CONSENT IS GIVEN. EG. COLLECTED FOR MEDICAL REASON BUT SHOULD NOT DISCLOSE TO INSURANCE. HOWEVER CAN USE FOR RELATED EVENTS OR FOR LAW ENFORCEMENT.
       • PRINCIPLE 7 - DIRECT MARKETING
         • CONSENT MUST BE GIVEN BY INDIVIDUAL. PERSON MUST BE INFORMED IF THIRD PARTY MARKETING
       • PRINCIPLE 8 - CROSS-BORDER DISCLOSURE
         • THE ENTITY MUST TAKE RESONALBLE STEPS SO OVERSEAS ENTITY DOES NOT BREACH APP,
       • PRINCIPLE 9 - USING GOVERNMENT RELATED IDENTIFIESR S
         • ORGS MUST NOT USE THE GOVE IDENTIFIERS AS ITS OWN . E..G DRIVER LICENSE. ORGS MUST NOT DISCLOSE IDENTIRIERS
     • Part 4 - Integrity of Personal Info
       • PRINCIPLE 10 - QUALITY OF PERSONAL INFO
         • THE DATA COLLECTED MUST BE ACURATE. ENSURE THAT REASONABLE STEPS TAKEN TO ENSURE THE INFORMATION SHARED IS OF QUALITY
       • PRINCIPLE 11- SECURITY OF PERSONAL INFO
         • ENTITIES MUST TAKE STEPS TO PREVENT MISUSE, INTERFERENCE, LOSS, UNAUTHORISED ACCESS/MODIFICATION/DISCLOSURE. INFO NEED TO BE DESTROYED IF NO NO LONGER NEEDED
     • Part 5 - Access and Correction of Personal Info
       • PRINICPLE 12 - ACCESS TO PERSONAL INFO
         • ACCESS - ENTITIY MUST PROVIDEACCESS TO HIS/HER OWN INFO
       • PRINCIPLE 13 - CORRECTION OF PERSONAL INFO
         • REASONABLE STEPS AND MECHANISMS SHOUDL BE PROVIDED TO CORRECT THE NFO

2. D2B - Data Warehousing Design - Humza

   • DW is NOT - product, language, project, data model or copy of OLTP
   • Analytical Eco System
   • Dimensional Modeling
     • Dimension - describe a business event, wide but shallow. conformed across the org
       • Attributes of a dimension - can be grouped
         • Determine the granularity
       • Directly relate to Fact table. Denormalised.
         • Surrogate key - created for DW linked to Natural Key of the source ssytem
           • Why you need Surrogate key? - Track history - slowly changing dimension - avoid conflicts of source key - insulates backend app changes i.e; if natural key changes it doesn't affect DW. Different backend apps use different columns as the key
       • Grain - Can be different grain for each fact
         • Lowest possible grain - drill down
     • Fact Table- Measurable metrics. More dimensions than facts. - Narrow but deep
     • Embedded Hierarchies- e.g. City > State > Country
     • Steps - Choose business process - declare grain - identify dimensions - identify the facts
       • Business process - Requirements + Data availability
       • Flavours of facts
     • CONFORMITY OF DIMENSIONS - essential for Enterprise DW
     • 7 Ws -How? • What ? • When? • Where? • Who? • How Many? • Why?
   • Slowly Changing Dimensions
     • 1. overwrite old values
     • 2. Keep the old values by creating a new row - need flags for current
     • 3. Keep old values in another colums. Need coding
   • Rapidly Changing Dimensions • Mini-dimensions • Degenerate Dimension

Imagens e arquivos do mapa