Dimensionality Reduction in Data Science

1. What is Data Science (DS)?1.1 Major Families of Data Science Problems1.1.1 Classification Problems1.1.2 Prediction Problems1.1.3 Clustering Problems1.2 Data, Big Data and Pre-processing1.2.1 What is Data?1.2.2 Big data1.2.3 Data Cleansing1.2.4 Data Visualization1.2.5 Data Understanding1.3 Populations and Data Sampling1.3.1 Sampling1.3.2 Training, Testing and Validation1.4 Overview and Scope1.4.1 Prerequisites and Layout1.4.2 Data Science Methodology1.4.3 Scope of the Book2. Solutions to Data Science Problems2.1 Conventional Statistical Solutions2.1.1 Linear Multiple Regression Model: Continuous Response2.1.2 Logistic Regression: Categorical Response2.1.3 Variable Selection and Model Building2.1.4 Generalized Linear Model (GLM)2.1.5 Decision Trees2.1.6 Bayesian Learning2.2 Machine Learning Solutions: Supervised2.2.1 k-Nearest Neighbors (kNN)2.2.2 Ensemble Methods2.2.3 Support Vector Machines (SVMs)2.2.4 Neural Networks (NNs)2.3 Machine Learning Solutions: Unsupervised2.3.1 Hard Clustering2.3.2 Soft Clustering2.4 Controls, Evaluation and Assessment2.4.1 Evaluation Methods2.4.2 Metrics for Assessment3. What is Dimensionality Reduction (DR)?3.1 Dimensionality Reduction3.2 Major Approaches to Dimensionality Reduction3.2.1 Conventional Statistical Approaches3.2.2 Geometric Approaches3.2.3 Information-theoretic Approaches3.2.4 Molecular Computing Approaches3.3 The Blessings of Dimensionality4. Conventional Statistical Approaches4.1 Principal Component Analysis (PCA)4.1.1 Obtaining the Principal Components4.1.2 Singular value decomposition (SVD)4.2 Nonlinear PCA 4.2.1 Kernel PCA4.2.2 Independent component analysis (ICA)4.3 Nonnegative Matrix Factorization (NMF)4.3.1 Approximate Solutions4.3.2 Clustering and Other Applications4.4 Discriminant Analysis4.4.1 Linear discriminant analysis (LDA)4.4.2 Quadratic discriminant analysis (QDA)4.5 Sliced Inverse Regression (SIR)5. Geometric Approaches5.1 Introduction to Manifolds5.2 Manifold Learning Methods5.2.1 Multi-Dimensional Scaling (MDS)5.2.2 Isometric Mapping (ISOMAP)5.2.3 t-Stochastic Neighbor Embedding ( t-SNE )5.3 Exploiting Randomness (RND)6. Information-theoretic Approaches6.1 Shannon Entropy (H)6.2 Reduction by Conditional Entropy6.3 Reduction by Iterated Conditional Entropy6.4 Reduction by Conditional Entropy on Targets6.5 Other Variations7. Molecular Computing Approaches7.1 Encoding Abiotic Data into DNA7.2 Deep Structure of DNA Spaces7.2.1 Structural Properties of DNA Spaces7.2.2 Noncrosshybridizing (nxh) Bases7.3 Reduction by Genomic Signatures7.3.1 Background7.3.2 Genomic Signatures7.4 Reduction by Pmeric Signatures8. Statistical Learning Approaches8.1 Reduction by Multiple Regression8.2 Reduction by Ridge Regression8.3 Reduction by Lasso Regression 8.4 Selection versus Shrinkage8.5 Further refinements9. Machine Learning Approaches9.1 Autoassociative Feature Encoders9.1.1 Undercomplete Autoencoders 9.1.2 Sparse Autoencoders9.1.3 Variational Autoencoders9.1.4 Dimensionality Reduction in MNIST