Data Warehousing and Mining

Data Warehousing and Mining

1. Decision-Support Systems

   • Database applications can be broadly classified into transaction-processing and decision-support systems.
     • Transaction-processing systems are systems that record information about transactions, such as product sales information for companies, or course registration and grade information for universities.
     • Decision-support systems help managers to decide what products to stock in a shop, what products to manufacture in a factory, or which of the applicants should be admitted to a university.
   • online analytical processing (OLAP).
   • There are several packages, such as SAS and S++, that help in statistical analysis.
   • The field of data mining combines knowledge- discovery techniques invented by artificial intelligence

2. Data Warehousing

   • Setting up queries on individual sources is both cumbersome and inefficient.
   • Corporate decision makers require access to information from multiple such sources.
   • Components of a Data Warehouse
     • When and how to gather data. In a source-driven architecture for gathering data, the data sources transmit new information, either continually (as transaction processing takes place), or periodically
     • Part of the task of a warehouse is to perform schema integration, and to convert data to the integrated schema before they are stored.
     • Thetaskofcorrectingandpreprocessing data is called data cleansing.
     • Updates on relations at the data sources must be propagated to the data warehouse.
   • Warehouse Schemas
     • Data warehouses typically have schemas that are designed for data analysis, using tools such as OLAP tools.
   • Column-Oriented Storage
     • in column-oriented storage, each attribute of a relation is stored in a separate file, with values from successive tuples stored at successive positions in the file

3. Data Mining

   • The term data mining refers loosely to the process of semiautomatically analyzing large databases to find useful patterns.
   • Like knowledge discovery in artificial intelligence (also called machine learning) or statistical analysis, data mining attempts to discover rules and patterns from data
   • Usually there is a manual component to data mining, consisting of preprocessing data to a form acceptable to the algorithms and postprocessing of discovered patterns to find novel ones that could be useful.

4. Classification

   • prediction is one of the most important types of data mining.
   • the classification problem is this: Given that items belong to one of several classes, and given past instances (called training instances)ofitems along with the classes to which they belong, the problem is to predict the class to which a new item belongs
   • Classification can be done by finding rules that partition the given data into disjoint groups.
   • Decision-Tree Classifiers
     • As the name suggests, decision-tree classifiers use a tree; each leaf node has an associated class, and each internal node has a predicate (or more generally, a function) associated with it.
   • Finding Best Splits
     • Attributes can be either continuous valued,thatis, the values can be ordered in a fashion meaningful to classification, such as age or income, or they can be categorical; that is, they have no meaningful order
       • Usually attributes that are numbers (integers/reals) are treated as continuous valued while character string attributes are treated as categorical, but this may be controlled by the user of the system
         • A subtree is overfitted if it has been so highly tuned to the specifics of the training data that it makes many classification errors on other data.
   • Regression
     • Regression deals with the prediction of a value, rather than a class.
     • In general, we wish to find a curve (defined by a polynomial or other formula) that fits the data

5. Association Rules

   • Association information can be used in several ways. When a customer buys a particular book, an online shop may suggest associated books.
   • Other Types of Associations
     • Using plain association rules has several shortcomings. One of the major shortcomings is that many associations are not very interesting, since they can be predicted
       • correlations can be positive, in that the co-occurrence is higher than would have been expected, or negative, in that the items co-occur less frequently than predicted.

6. Clustering

   • Intuitively, clustering refers to the problem of finding clusters of points in the given data.
   • The problem of clustering can be formalized from distance metrics in several ways. One way is to phrase it as the problem of grouping points into k sets (for a given k) so that the average distance of points from the centroid of their assigned cluster is minimized.
   • Other Forms of Data Mining
     • Text mining applies data-mining techniques to textual documents.
     • Data-visualization systems help users to examine large volumes of data, and to detect patterns visually.

7. Nombre: Juan Manuel Figueroa Solís Carnet: 2018139700 Universidad: I.T.C.R Curso: IC4302 Bases de Datos 2 Fecha: 12/10/21 Profesor: Erick Hernández Bonilla Grupo: 40

Bilder und Dateien der Mindmap