iot+ml

iot+ml

1. big data

   • dimensions
     •  Characteristics of big data There are more Vs See: The anatomy of big data computing,
     • volume
       • Amount, size, and scale of the data. In ML, size is the number of SAMPLES or FEATURES. Small complex data is compared to Large simple data.
       • processing performance
         • large scale -> large complexity
         • data size affects performance
           • Resilient Distributed Datasets can help
         • manipulations for big data
           • data manipulations
             • Modify the data in order to mimic non-Big Data. Problems: high dimensionality (widedatasets) and large number of samples (high datasets).
             • dimensionality reduction
               • Aims to map high dimensional-ity space onto lower-dimensionality one without significant loss of information.
               • Principal Component Analisys
                 • One popular, but very old technique (it originates from 1901) is principal component analysis (PCA). PCA belongs to the family of linear mapping techniques: orthogonal transformations are applied to transform a set of possibly correlated variables into a set of linearly uncorrelated variables, called principal components. The first principal component accounts for the largest proportion of the variability in the data, the second one has the next highest variance and is orthogonal to the first, and so on. Thus, choosing only the first p principal components can reduce dimensionality.
               • kernel PCA
               • Laplacian Eigenmaps
               • Locally Linear Embedding LLE
               • Hessian LLE
               • Random Projection
               • Autoencoders
             • instance selection
             • data cleaning
       • curse of dimensionality
     • velocity
     • variety
     • veracity

2. machine learning

   • more data != better results