SQL on Big Data ― Technology, Architecture, and Innovation

This book explains various commercial and open source products that perform SQL on Hadoop and big data. It equips the reader with an understanding of the various architectures being used and how the tools work internally in terms of execution, data movement, latency, scalability, performance, and system requirements.

SQL on Big Data consolidates in one place solutions to the challenges associated with the requirements of speed, scalability, and variety of operations needed for data integration and SQL operations. After discussing the history of how and why SQL on Hadoop and big data provides the best way to look at data, the book offers an understanding of the direction of this rapidly evolving space.

SQL on Big Data lays out the picture of the road ahead—what capabilities are on the horizon and how do they solve the issues of performance and scalability and the ability to handle different data types? The book covers how SQL on big data is permeating the OLTP, OLAP, and operational analytics space and rapidly evolving HTAP systems.

The book contains detailed coverage of:

• Batch architectures—an understanding of the internals and how the existing Hive engine is evolving.
  
• Interactive architectures—an understanding of how systems are architected and how they solve problems using MPP and indexes.
  
• Streaming architectures—understanding of how the systems are constructed and how they solve problems using in-memory and lock-free data structures.

Sumit Pal is a big data and data science consultant working with multiple clients and advising them on their data architectures and big data solutions as well as providing hands on coding with Spark, Scala, Java and Python. He is a big data, visualization and data science consultant, and a software architect and big data enthusiast and builds end-to-end data-driven analytic systems. He has more than 22 years of experience in the software industry in various roles spanning companies from startups to enterprises.

Sumit has worked for Microsoft (SQL server development team), Oracle (OLAP development team) and Verizon (big data analytics team) in a career spanning 22 years. He has extensive experience in building scalable systems across the stack from middle-tier, data tier to visualization for analytics applications, using big data, and NoSQL DB. Sumit has deep expertise in database Internals, data warehouses, dimensional modeling, data science with Java and Python, and SQL.

Sumit started his career with SQL Server Development Team at Microsoft and then as a core server engineer for Oracle Corporation on their OLAP development team. He has also worked at Verizon as an Associate Director for big data architecture, where he strategized, managed, architected, and developed platforms and solutions for analytics and machine learning applications. Sumit has also served as Chief Architect at ModelN/LeapfrogRX, where he architected the middle tier core analytics platform with open source OLAP engine (Mondrian) on J2EE and solved some complex dimensional ETL, modelling, and performance optimization problems.