HADOOP

HADOOP

1. que es

   • Hadoop is an open source software infrastructure that provides highly reliable distributed processing of large data sets using simple programming models. Hadoop, known for its scalability, is based on commodity computer clusters, providing a cost-effective solution for storing and processing massive amounts of structured, semi-structured and unstructured data without format requirements.

2. characteristics

   • Open source: By having a characteristic so fundamental that allows this to be of form free, its source code is sample always available for anyone who interprets it since This allows you to perform some modifications according to user needs
   • Highly Escalating Cluster Being a very characteristic fundamental since it derives a large amount of data and information in a cluster that process in parallel
   • Fault tolerance When using this software basic may fail in any moment, to correct this hadoop uses datanodes which in turn manage a cluster that always guarantees availability of the data
   • Provides high Availability Bearing very much in mind possible errors or failures haddop makes use of the cluster function due to fault tolerance within the properties of it
   • Profitable for how big data works manages in its essence a basic hardware since previously the companies had large amount of information
   • Provides flexibility Designed to handle any type of data structured, semi structured and not structured,
   • Easy to use Being managed by processing of the information is not needs a expert knowledge for the use of the same which is easy to access
   • Use data locality Considering the location need of the information managed search system fast that facilitates access to information requested

3. HADOOP ENVIRONMENT

   • Being the owner of open source has revolutionized the field of data management. he hadoop environment is a feature essential as it guarantees optimal performance and efficient in data processing.
     • El hdfs
     • Map reduce
     • Yarn
     • Proper configuration
     • Data localization
     • Insufficient data collection
     • Inadequate cluster size
     • Ignore data security

4. CHRONOLOGY OF HADOOP DEVELOPMENT

   • 2002 was created as a process of engine system search
   • 2003 web crawling and indexing processes in order to preserve the information
   • n 2004 the mapreduce application
   • 2005 the platform for data brought limitations such as use maximum of 40 nodes
   • 2006 the association with the yahoo platform network it became possiblee
   • 2007 yahoo was the first platform that I achieve successfully test Hadoop
   • of 2008 appears first cluster with 4000 nodes
   • 2009 is achieved successfully the ordering of a PB (peta byte)
   • 2011 apache software foundation launched apache hadoop version 1.0
   • 2013 was launched new version 2.0.6
   • 2017 and currently the application has with apache hadoop application version 3.0

5. ADVANTAGES OF HADOOP

   • 1) Isolate developers from all difficulties present in parallel programming 2) It has a ecosystem that is of great help to the user since allows you to distribute the ones that are nothing more than computers with commodity – hardware 3) It is capable of run processes in parallel at all times 4) It has control modules for monitoring of the data 5) Presents an option that allows you to perform queries 6) Enhances the appearance of different add-ons which makes handling work easier

6. GOOGLE FILE SYSTEM

   • It is a distributed file system developed by Google inc. Which supports your entire IT infrastructure in a cloud, was created in order to efficiently find when searching for some data and information through one big data system cluster per system in parallel
     • SHADOOP FILE SYSTEM
       • It is an open source framework used to store and process efficiently set of massive data, which manage volumes ranging from gigabytes to petabytes of information data. For this reason in topics haddop efficiency handles the ability to create multiple clusters, from different computers to process and store data in parallel and with faster

7. MAPREDUCE IN HADOOP.

   • It is a software framework and programming model which is used to access and process large amounts of data. Mapreduce is a program that For its operation, it works in two phases, namely mapping and reduce. Mapping overheads deal with the division and data mapping, while reduction tasks mix and reduce data

8. RELATIONSHIP OF BIG DATA ANALYTICS WITH MACHINE LEARNING AND CLOUD COMPUITING

   • with some important topic, makes handle large volumes information are super massive at the time to group together the tools of Traditional processing is not competent to handle them, in addition to the speed at which they move through the network and their variety make its manipulation is complex, with this need arises for big data to give solves the manipulation of large volumes of information so profitable, effective and efficient having its field of action in companies systems health and many sectors that manage information by magnetic means thus entering the data environment analyticalthics analysis system information as a result of the study of big data in the cloud