Databricks Data Intelligence Platform
Author: Nikhil Gupta
Publisher: Springer Nature
Published:
Total Pages: 481
ISBN-13:
DOWNLOAD EBOOKPosts
Building the Data Lakehouse
Author: Bill Inmon
Publisher: Technics Publications
Published: 2021-10
Total Pages: 256
ISBN-13: 9781634629669
DOWNLOAD EBOOKThe data lakehouse is the next generation of the data warehouse and data lake, designed to meet today's complex and ever-changing analytics, machine learning, and data science requirements. Learn about the features and architecture of the data lakehouse, along with its powerful analytical infrastructure. Appreciate how the universal common connector blends structured, textual, analog, and IoT data. Maintain the lakehouse for future generations through Data Lakehouse Housekeeping and Data Future-proofing. Know how to incorporate the lakehouse into an existing data governance strategy. Incorporate data catalogs, data lineage tools, and open source software into your architecture to ensure your data scientists, analysts, and end users live happily ever after.
Distributed Data Systems with Azure Databricks
Author: Alan Bernardo Palacio
Publisher: Packt Publishing Ltd
Published: 2021-05-25
Total Pages: 414
ISBN-13: 1838642692
DOWNLOAD EBOOKQuickly build and deploy massive data pipelines and improve productivity using Azure Databricks Key FeaturesGet to grips with the distributed training and deployment of machine learning and deep learning modelsLearn how ETLs are integrated with Azure Data Factory and Delta LakeExplore deep learning and machine learning models in a distributed computing infrastructureBook Description Microsoft Azure Databricks helps you to harness the power of distributed computing and apply it to create robust data pipelines, along with training and deploying machine learning and deep learning models. Databricks' advanced features enable developers to process, transform, and explore data. Distributed Data Systems with Azure Databricks will help you to put your knowledge of Databricks to work to create big data pipelines. The book provides a hands-on approach to implementing Azure Databricks and its associated methodologies that will make you productive in no time. Complete with detailed explanations of essential concepts, practical examples, and self-assessment questions, you’ll begin with a quick introduction to Databricks core functionalities, before performing distributed model training and inference using TensorFlow and Spark MLlib. As you advance, you’ll explore MLflow Model Serving on Azure Databricks and implement distributed training pipelines using HorovodRunner in Databricks. Finally, you’ll discover how to transform, use, and obtain insights from massive amounts of data to train predictive models and create entire fully working data pipelines. By the end of this MS Azure book, you’ll have gained a solid understanding of how to work with Databricks to create and manage an entire big data pipeline. What you will learnCreate ETLs for big data in Azure DatabricksTrain, manage, and deploy machine learning and deep learning modelsIntegrate Databricks with Azure Data Factory for extract, transform, load (ETL) pipeline creationDiscover how to use Horovod for distributed deep learningFind out how to use Delta Engine to query and process data from Delta LakeUnderstand how to use Data Factory in combination with DatabricksUse Structured Streaming in a production-like environmentWho this book is for This book is for software engineers, machine learning engineers, data scientists, and data engineers who are new to Azure Databricks and want to build high-quality data pipelines without worrying about infrastructure. Knowledge of Azure Databricks basics is required to learn the concepts covered in this book more effectively. A basic understanding of machine learning concepts and beginner-level Python programming knowledge is also recommended.
Optimizing Databricks Workloads
Author: Anirudh Kala
Publisher: Packt Publishing Ltd
Published: 2021-12-24
Total Pages: 230
ISBN-13: 180181192X
DOWNLOAD EBOOKAccelerate computations and make the most of your data effectively and efficiently on Databricks Key FeaturesUnderstand Spark optimizations for big data workloads and maximizing performanceBuild efficient big data engineering pipelines with Databricks and Delta LakeEfficiently manage Spark clusters for big data processingBook Description Databricks is an industry-leading, cloud-based platform for data analytics, data science, and data engineering supporting thousands of organizations across the world in their data journey. It is a fast, easy, and collaborative Apache Spark-based big data analytics platform for data science and data engineering in the cloud. In Optimizing Databricks Workloads, you will get started with a brief introduction to Azure Databricks and quickly begin to understand the important optimization techniques. The book covers how to select the optimal Spark cluster configuration for running big data processing and workloads in Databricks, some very useful optimization techniques for Spark DataFrames, best practices for optimizing Delta Lake, and techniques to optimize Spark jobs through Spark core. It contains an opportunity to learn about some of the real-world scenarios where optimizing workloads in Databricks has helped organizations increase performance and save costs across various domains. By the end of this book, you will be prepared with the necessary toolkit to speed up your Spark jobs and process your data more efficiently. What you will learnGet to grips with Spark fundamentals and the Databricks platformProcess big data using the Spark DataFrame API with Delta LakeAnalyze data using graph processing in DatabricksUse MLflow to manage machine learning life cycles in DatabricksFind out how to choose the right cluster configuration for your workloadsExplore file compaction and clustering methods to tune Delta tablesDiscover advanced optimization techniques to speed up Spark jobsWho this book is for This book is for data engineers, data scientists, and cloud architects who have working knowledge of Spark/Databricks and some basic understanding of data engineering principles. Readers will need to have a working knowledge of Python, and some experience of SQL in PySpark and Spark SQL is beneficial.
Architecting Modern Data Platforms
Author: Jan Kunigk
Publisher: "O'Reilly Media, Inc."
Published: 2018-12-05
Total Pages: 688
ISBN-13: 1491969229
DOWNLOAD EBOOKThere’s a lot of information about big data technologies, but splicing these technologies into an end-to-end enterprise data platform is a daunting task not widely covered. With this practical book, you’ll learn how to build big data infrastructure both on-premises and in the cloud and successfully architect a modern data platform. Ideal for enterprise architects, IT managers, application architects, and data engineers, this book shows you how to overcome the many challenges that emerge during Hadoop projects. You’ll explore the vast landscape of tools available in the Hadoop and big data realm in a thorough technical primer before diving into: Infrastructure: Look at all component layers in a modern data platform, from the server to the data center, to establish a solid foundation for data in your enterprise Platform: Understand aspects of deployment, operation, security, high availability, and disaster recovery, along with everything you need to know to integrate your platform with the rest of your enterprise IT Taking Hadoop to the cloud: Learn the important architectural aspects of running a big data platform in the cloud while maintaining enterprise security and high availability
Winning with Data
Author: Tomasz Tunguz
Publisher: John Wiley & Sons
Published: 2016-06-20
Total Pages: 179
ISBN-13: 1119257239
DOWNLOAD EBOOKCrest the data wave with a deep cultural shift Winning with Data explores the cultural changes big data brings to business, and shows you how to adapt your organization to leverage data to maximum effect. Authors Tomasz Tunguz and Frank Bien draw on extensive background in big data, business intelligence, and business strategy to provide a blueprint for companies looking to move head-on into the data wave. Instrumentation is discussed in detail, but the core of the change is in the culture—this book provides sound guidance on building the type of organizational culture that creates and leverages data daily, in every aspect of the business. Real-world examples illustrate these important concepts at work: you'll learn how data helped Warby-Parker disrupt a $13 billion monopolized market, how ThredUp uses data to process more than 20 thousand items of clothing every day, how Venmo leverages data to build better products, how HubSpot empowers their salespeople to be more productive, and more. From decision making and strategy to shipping and sales, this book shows you how data makes better business. Big data has taken on buzzword status, but there is little real guidance for companies seeking everyday business data solutions. This book takes a deeper look at big data in business, and shows you how to shift internal culture ahead of the curve. Understand the changes a data culture brings to companies Instrument your company for maximum benefit Utilize data to optimize every aspect of your business Improve decision making and transform business strategy Big data is becoming the number-one topic in business, yet no one is asking the right questions. Leveraging the full power of data requires more than good IT—organization-wide buy-in is essential for long-term success. Winning with Data is the expert guide to making data work for your business, and your needs.
Data Mesh
Author: Zhamak Dehghani
Publisher: "O'Reilly Media, Inc."
Published: 2022-03-08
Total Pages: 387
ISBN-13: 1492092363
DOWNLOAD EBOOKMany enterprises are investing in a next-generation data lake, hoping to democratize data at scale to provide business insights and ultimately make automated intelligent decisions. In this practical book, author Zhamak Dehghani reveals that, despite the time, money, and effort poured into them, data warehouses and data lakes fail when applied at the scale and speed of today's organizations. A distributed data mesh is a better choice. Dehghani guides architects, technical leaders, and decision makers on their journey from monolithic big data architecture to a sociotechnical paradigm that draws from modern distributed architecture. A data mesh considers domains as a first-class concern, applies platform thinking to create self-serve data infrastructure, treats data as a product, and introduces a federated and computational model of data governance. This book shows you why and how. Examine the current data landscape from the perspective of business and organizational needs, environmental challenges, and existing architectures Analyze the landscape's underlying characteristics and failure modes Get a complete introduction to data mesh principles and its constituents Learn how to design a data mesh architecture Move beyond a monolithic data lake to a distributed data mesh.
Data Warehouse Performance
Author: W. H. Inmon
Publisher:
Published: 1998-11-13
Total Pages: 466
ISBN-13:
DOWNLOAD EBOOKReduce operating and maintenance costs while substantially improving the performance of new and existing data warehouses and data marts Data Warehouse Performance This book tells you what you need to know to design, build, and manage data warehouses and data marts for optimum performance. Written by an all-star team of data warehouse pioneers and innovators-including Bill Inmon, "the father of the data warehouse," and Ken Rudin, one of the leading experts on performance-the book describes the layers of a high-performance data warehouse environment and guides the reader through their implementation and management. It also supplies proven techniques for supercharging the performance of existing environments. Crucial topics covered include: * Mitigating the impact of dormant data on performance * Data cleansing and implementation techniques * Implementing platform components like data marts to support scalability * Database design, sizing, and optimization techniques, including star schema and indexing * Hardware assessment, selection, and sizing * The role of monitors in balancing workload and assessing performance * Creating a service management contract to meet user expectations
Data Lakehouse in Action
Author: Pradeep Menon
Publisher: Packt Publishing Ltd
Published: 2022-03-17
Total Pages: 206
ISBN-13: 1801815100
DOWNLOAD EBOOKPropose a new scalable data architecture paradigm, Data Lakehouse, that addresses the limitations of current data architecture patterns Key FeaturesUnderstand how data is ingested, stored, served, governed, and secured for enabling data analyticsExplore a practical way to implement Data Lakehouse using cloud computing platforms like AzureCombine multiple architectural patterns based on an organization's needs and maturity levelBook Description The Data Lakehouse architecture is a new paradigm that enables large-scale analytics. This book will guide you in developing data architecture in the right way to ensure your organization's success. The first part of the book discusses the different data architectural patterns used in the past and the need for a new architectural paradigm, as well as the drivers that have caused this change. It covers the principles that govern the target architecture, the components that form the Data Lakehouse architecture, and the rationale and need for those components. The second part deep dives into the different layers of Data Lakehouse. It covers various scenarios and components for data ingestion, storage, data processing, data serving, analytics, governance, and data security. The book's third part focuses on the practical implementation of the Data Lakehouse architecture in a cloud computing platform. It focuses on various ways to combine the Data Lakehouse pattern to realize macro-patterns, such as Data Mesh and Data Hub-Spoke, based on the organization's needs and maturity level. The frameworks introduced will be practical and organizations can readily benefit from their application. By the end of this book, you'll clearly understand how to implement the Data Lakehouse architecture pattern in a scalable, agile, and cost-effective manner. What you will learnUnderstand the evolution of the Data Architecture patterns for analyticsBecome well versed in the Data Lakehouse pattern and how it enables data analyticsFocus on methods to ingest, process, store, and govern data in a Data Lakehouse architectureLearn techniques to serve data and perform analytics in a Data Lakehouse architectureCover methods to secure the data in a Data Lakehouse architectureImplement Data Lakehouse in a cloud computing platform such as AzureCombine Data Lakehouse in a macro-architecture pattern such as Data MeshWho this book is for This book is for data architects, big data engineers, data strategists and practitioners, data stewards, and cloud computing practitioners looking to become well-versed with modern data architecture patterns to enable large-scale analytics. Basic knowledge of data architecture and familiarity with data warehousing concepts are required.
SAP HANA 2.0
Author: Denys Van Kempen
Publisher: SAP PRESS
Published: 2019
Total Pages: 438
ISBN-13: 9781493218387
DOWNLOAD EBOOKEnter the fast-paced world of SAP HANA 2.0 with this introductory guide. Begin with an exploration of the technological backbone of SAP HANA as a database and platform. Then, step into key SAP HANA user roles and discover core capabilities for administration, application development, advanced analytics, security, data integration, and more. No matter how SAP HANA 2.0 fits into your business, this book is your starting point. In this book, you'll learn about: a. Technology Discover what makes an in-memory database platform. Learn about SAP HANA's journey from version 1.0 to 2.0, take a tour of your technology options, and walk through deployment scenarios and implementation requirements. b. Tools Unpack your SAP HANA toolkit. See essential tools in action, from SAP HANA cockpit and SAP HANA studio, to the SAP HANA Predictive Analytics Library and SAP HANA smart data integration. c. Key Roles Understand how to use SAP HANA as a developer, administrator, data scientist, data center architect, and more. Explore key tasks like backend programming with SQLScript, security setup with roles and authorizations, data integration with the SAP HANA Data Management Suite, and more. Highlights include: 1) Architecture 2) Administration 3) Application development 4) Analytics 5) Security 6) Data integration 7) Data architecture 8) Data center
