Hadoop 2.x Administration Cookbook

Hadoop 2.x Administration Cookbook

Author: Gurmukh Singh

Publisher: Packt Publishing Ltd

Published: 2017-05-26

Total Pages: 348

ISBN-13: 1787126870

DOWNLOAD EBOOK

Over 100 practical recipes to help you become an expert Hadoop administrator About This Book Become an expert Hadoop administrator and perform tasks to optimize your Hadoop Cluster Import and export data into Hive and use Oozie to manage workflow. Practical recipes will help you plan and secure your Hadoop cluster, and make it highly available Who This Book Is For If you are a system administrator with a basic understanding of Hadoop and you want to get into Hadoop administration, this book is for you. It's also ideal if you are a Hadoop administrator who wants a quick reference guide to all the Hadoop administration-related tasks and solutions to commonly occurring problems What You Will Learn Set up the Hadoop architecture to run a Hadoop cluster smoothly Maintain a Hadoop cluster on HDFS, YARN, and MapReduce Understand high availability with Zookeeper and Journal Node Configure Flume for data ingestion and Oozie to run various workflows Tune the Hadoop cluster for optimal performance Schedule jobs on a Hadoop cluster using the Fair and Capacity scheduler Secure your cluster and troubleshoot it for various common pain points In Detail Hadoop enables the distributed storage and processing of large datasets across clusters of computers. Learning how to administer Hadoop is crucial to exploit its unique features. With this book, you will be able to overcome common problems encountered in Hadoop administration. The book begins with laying the foundation by showing you the steps needed to set up a Hadoop cluster and its various nodes. You will get a better understanding of how to maintain Hadoop cluster, especially on the HDFS layer and using YARN and MapReduce. Further on, you will explore durability and high availability of a Hadoop cluster. You'll get a better understanding of the schedulers in Hadoop and how to configure and use them for your tasks. You will also get hands-on experience with the backup and recovery options and the performance tuning aspects of Hadoop. Finally, you will get a better understanding of troubleshooting, diagnostics, and best practices in Hadoop administration. By the end of this book, you will have a proper understanding of working with Hadoop clusters and will also be able to secure, encrypt it, and configure auditing for your Hadoop clusters. Style and approach This book contains short recipes that will help you run a Hadoop cluster efficiently. The recipes are solutions to real-life problems that administrators encounter while working with a Hadoop cluster


Apache Hadoop 3 Quick Start Guide

Apache Hadoop 3 Quick Start Guide

Author: Hrishikesh Vijay Karambelkar

Publisher: Packt Publishing Ltd

Published: 2018-10-31

Total Pages: 214

ISBN-13: 1788994345

DOWNLOAD EBOOK

A fast paced guide that will help you learn about Apache Hadoop 3 and its ecosystem Key FeaturesSet up, configure and get started with Hadoop to get useful insights from large data setsWork with the different components of Hadoop such as MapReduce, HDFS and YARN Learn about the new features introduced in Hadoop 3Book Description Apache Hadoop is a widely used distributed data platform. It enables large datasets to be efficiently processed instead of using one large computer to store and process the data. This book will get you started with the Hadoop ecosystem, and introduce you to the main technical topics, including MapReduce, YARN, and HDFS. The book begins with an overview of big data and Apache Hadoop. Then, you will set up a pseudo Hadoop development environment and a multi-node enterprise Hadoop cluster. You will see how the parallel programming paradigm, such as MapReduce, can solve many complex data processing problems. The book also covers the important aspects of the big data software development lifecycle, including quality assurance and control, performance, administration, and monitoring. You will then learn about the Hadoop ecosystem, and tools such as Kafka, Sqoop, Flume, Pig, Hive, and HBase. Finally, you will look at advanced topics, including real time streaming using Apache Storm, and data analytics using Apache Spark. By the end of the book, you will be well versed with different configurations of the Hadoop 3 cluster. What you will learnStore and analyze data at scale using HDFS, MapReduce and YARNInstall and configure Hadoop 3 in different modesUse Yarn effectively to run different applications on Hadoop based platformUnderstand and monitor how Hadoop cluster is managedConsume streaming data using Storm, and then analyze it using SparkExplore Apache Hadoop ecosystem components, such as Flume, Sqoop, HBase, Hive, and KafkaWho this book is for Aspiring Big Data professionals who want to learn the essentials of Hadoop 3 will find this book to be useful. Existing Hadoop users who want to get up to speed with the new features introduced in Hadoop 3 will also benefit from this book. Having knowledge of Java programming will be an added advantage.


Creativity in Intelligent Technologies and Data Science

Creativity in Intelligent Technologies and Data Science

Author: Alla G. Kravets

Publisher: Springer Nature

Published: 2021-09-15

Total Pages: 627

ISBN-13: 3030870340

DOWNLOAD EBOOK

This book constitutes the proceedings of the 4th Conference on Creativity in Intellectual Technologies and Data Science, CIT&DS 2021, held in Volgograd, Russia, in September 2021. The 39 full papers, 7 short papers, and 2 keynote papers presented were carefully reviewed and selected from 182 submissions. The papers are organized in the following topical sections: Artificial intelligence and deep learning technologies: knowledge discovery in patent and open sources; open science semantic technologies; IoT and computer vision in knowledge-based control; Cyber-physical systems and big data-driven control: pro-active modeling in intelligent decision making support; design creativity in CASE/CAI/CAD/PDM; intelligent technologies in urban design and computing; Intelligent technologies in social engineering: data science in social networks analysis and cyber security; educational creativity and game-based learning; intelligent assistive technologies: software design and application.


Spark SQL 2.x Fundamentals and Cookbook

Spark SQL 2.x Fundamentals and Cookbook

Author: HadoopExam Learning Resources

Publisher: HadoopExam Learning Resources

Published: 2018-09-02

Total Pages: 162

ISBN-13:

DOWNLOAD EBOOK

Apache Spark is one of the fastest growing technology in BigData computing world. It support multiple programming languages like Java, Scala, Python and R. Hence, many existing and new framework started to integrate Spark platform as well in their platform e.g. Hadoop, Cassandra, EMR etc. While creating Spark certification material HadoopExam technical team found that there is no proper material and book is available for the Spark SQL (version 2.x) which covers the concepts as well as use of various features and found difficulty in creating the material. Therefore, they decided to create full length book for Spark SQL and outcome of that is this book. In this book technical team try to cover both fundamental concepts of Spark SQL engine and many exercises approx. 35+ so that most of the programming features can be covered. There are approximately 35 exercises and total 15 chapters which covers the programming aspects of SparkSQL. All the exercises given in this book are written using Scala. However, concepts remain same even if you are using different programming language. This book is good for following audiance - Data scientists - Spark Developer - Data Engineer - Data Analytics - Java/Python Developer - Scala Developer


Hadoop Real-World Solutions Cookbook

Hadoop Real-World Solutions Cookbook

Author: Tanmay Deshpande

Publisher: Packt Publishing Ltd

Published: 2016-03-31

Total Pages: 290

ISBN-13: 1784398004

DOWNLOAD EBOOK

Over 90 hands-on recipes to help you learn and master the intricacies of Apache Hadoop 2.X, YARN, Hive, Pig, Oozie, Flume, Sqoop, Apache Spark, and Mahout About This Book Implement outstanding Machine Learning use cases on your own analytics models and processes. Solutions to common problems when working with the Hadoop ecosystem. Step-by-step implementation of end-to-end big data use cases. Who This Book Is For Readers who have a basic knowledge of big data systems and want to advance their knowledge with hands-on recipes. What You Will Learn Installing and maintaining Hadoop 2.X cluster and its ecosystem. Write advanced Map Reduce programs and understand design patterns. Advanced Data Analysis using the Hive, Pig, and Map Reduce programs. Import and export data from various sources using Sqoop and Flume. Data storage in various file formats such as Text, Sequential, Parquet, ORC, and RC Files. Machine learning principles with libraries such as Mahout Batch and Stream data processing using Apache Spark In Detail Big data is the current requirement. Most organizations produce huge amount of data every day. With the arrival of Hadoop-like tools, it has become easier for everyone to solve big data problems with great efficiency and at minimal cost. Grasping Machine Learning techniques will help you greatly in building predictive models and using this data to make the right decisions for your organization. Hadoop Real World Solutions Cookbook gives readers insights into learning and mastering big data via recipes. The book not only clarifies most big data tools in the market but also provides best practices for using them. The book provides recipes that are based on the latest versions of Apache Hadoop 2.X, YARN, Hive, Pig, Sqoop, Flume, Apache Spark, Mahout and many more such ecosystem tools. This real-world-solution cookbook is packed with handy recipes you can apply to your own everyday issues. Each chapter provides in-depth recipes that can be referenced easily. This book provides detailed practices on the latest technologies such as YARN and Apache Spark. Readers will be able to consider themselves as big data experts on completion of this book. This guide is an invaluable tutorial if you are planning to implement a big data warehouse for your business. Style and approach An easy-to-follow guide that walks you through world of big data. Each tool in the Hadoop ecosystem is explained in detail and the recipes are placed in such a manner that readers can implement them sequentially. Plenty of reference links are provided for advanced reading.


DataBricks® PySpark 2.x Certification Practice Questions

DataBricks® PySpark 2.x Certification Practice Questions

Author:

Publisher: HadoopExam Learning Resources

Published:

Total Pages: 183

ISBN-13:

DOWNLOAD EBOOK

This book contains the questions answers and some FAQ about the Databricks Spark Certification for version 2.x, which is the latest release from Apache Spark. In this book we will be having in total 75 practice questions. Almost all required question would have in detail explanation to the questions and answers, wherever required. Don’t consider this book as a guide, it is more of question and answer practice book. This book also give some references as well like how to prepare further to ensure that you clear the certification exam. This book will particularly focus on the Python version of the certification preparation material. Please note these are practice questions and not dumps, hence just memorizing the question and answers will not help in the real exam. You need to understand the concepts in detail as well as you should be able to solve the programming questions at the end in real worlds work you should be able to write code using PySpark whether you are Data Engineer, Data Analytics Engineer, Data Scientists or Programmer. Hence, take the opportunity to learn each question and also go through the explanation of the questions.


Apache Spark 2.x Machine Learning Cookbook

Apache Spark 2.x Machine Learning Cookbook

Author: Siamak Amirghodsi

Publisher: Packt Publishing Ltd

Published: 2017-09-22

Total Pages: 658

ISBN-13: 1782174605

DOWNLOAD EBOOK

Simplify machine learning model implementations with Spark About This Book Solve the day-to-day problems of data science with Spark This unique cookbook consists of exciting and intuitive numerical recipes Optimize your work by acquiring, cleaning, analyzing, predicting, and visualizing your data Who This Book Is For This book is for Scala developers with a fairly good exposure to and understanding of machine learning techniques, but lack practical implementations with Spark. A solid knowledge of machine learning algorithms is assumed, as well as hands-on experience of implementing ML algorithms with Scala. However, you do not need to be acquainted with the Spark ML libraries and ecosystem. What You Will Learn Get to know how Scala and Spark go hand-in-hand for developers when developing ML systems with Spark Build a recommendation engine that scales with Spark Find out how to build unsupervised clustering systems to classify data in Spark Build machine learning systems with the Decision Tree and Ensemble models in Spark Deal with the curse of high-dimensionality in big data using Spark Implement Text analytics for Search Engines in Spark Streaming Machine Learning System implementation using Spark In Detail Machine learning aims to extract knowledge from data, relying on fundamental concepts in computer science, statistics, probability, and optimization. Learning about algorithms enables a wide range of applications, from everyday tasks such as product recommendations and spam filtering to cutting edge applications such as self-driving cars and personalized medicine. You will gain hands-on experience of applying these principles using Apache Spark, a resilient cluster computing system well suited for large-scale machine learning tasks. This book begins with a quick overview of setting up the necessary IDEs to facilitate the execution of code examples that will be covered in various chapters. It also highlights some key issues developers face while working with machine learning algorithms on the Spark platform. We progress by uncovering the various Spark APIs and the implementation of ML algorithms with developing classification systems, recommendation engines, text analytics, clustering, and learning systems. Toward the final chapters, we'll focus on building high-end applications and explain various unsupervised methodologies and challenges to tackle when implementing with big data ML systems. Style and approach This book is packed with intuitive recipes supported with line-by-line explanations to help you understand how to optimize your work flow and resolve problems when working with complex data modeling tasks and predictive algorithms. This is a valuable resource for data scientists and those working on large scale data projects.


Apache Cassandra Certification Practice Material : 2019

Apache Cassandra Certification Practice Material : 2019

Author:

Publisher: HadoopExam Learning Resources

Published:

Total Pages: 120

ISBN-13:

DOWNLOAD EBOOK

About Professional Certification of Apache Cassandra: Apache Cassandra is one of the most popular NoSQL Database currently being used by many of the organization, globally in every industry like Aviation, Finance, Retail, Social Networking etc. It proves that there is quite a huge demand for certified Cassandra professionals. Having certification make your selection in the company make much easier. This certification is conducted by the DataStax®, which has the Enterprise Version of the Apache Cassandra and Leader in providing support for the open source Apache Cassandra NoSQL database. Cassandra is one of the Unique NoSQL Database. So go for its certification, it will certainly help in - Getting the Job - Increase in your salary - Growth in your career. - Managing Tera Bytes of Data. - Learning Distributed Database - Using CQL (Cassandra Query Language) Cassandra Certification Information: - Number of questions: 60 Multiple Choice - Time allowed in minutes: 90 - Required passing score: 75% - Languages: English Exam Objectives: There are in total 5 sections and you will be asked total 60 questions in real exam. Please check each section below with regards to the exam objective 1. Apache Cassandra™ data modeling 2. Fundamentals of replication and consistency 3. The distributed and internal architecture of Apache Cassandra™ 4. Installation and configuration 5. Basic tooling


Apache Sqoop Cookbook

Apache Sqoop Cookbook

Author: Kathleen Ting

Publisher: "O'Reilly Media, Inc."

Published: 2013-07-02

Total Pages: 125

ISBN-13: 1449364586

DOWNLOAD EBOOK

Integrating data from multiple sources is essential in the age of big data, but it can be a challenging and time-consuming task. This handy cookbook provides dozens of ready-to-use recipes for using Apache Sqoop, the command-line interface application that optimizes data transfers between relational databases and Hadoop. Sqoop is both powerful and bewildering, but with this cookbook’s problem-solution-discussion format, you’ll quickly learn how to deploy and then apply Sqoop in your environment. The authors provide MySQL, Oracle, and PostgreSQL database examples on GitHub that you can easily adapt for SQL Server, Netezza, Teradata, or other relational systems. Transfer data from a single database table into your Hadoop ecosystem Keep table data and Hadoop in sync by importing data incrementally Import data from more than one database table Customize transferred data by calling various database functions Export generated, processed, or backed-up data from Hadoop to your database Run Sqoop within Oozie, Hadoop’s specialized workflow scheduler Load data into Hadoop’s data warehouse (Hive) or database (HBase) Handle installation, connection, and syntax issues common to specific database vendors


Hadoop MapReduce v2 Cookbook - Second Edition

Hadoop MapReduce v2 Cookbook - Second Edition

Author: Thilina Gunarathne

Publisher: Packt Publishing Ltd

Published: 2015-02-25

Total Pages: 322

ISBN-13: 1783285486

DOWNLOAD EBOOK

If you are a Big Data enthusiast and wish to use Hadoop v2 to solve your problems, then this book is for you. This book is for Java programmers with little to moderate knowledge of Hadoop MapReduce. This is also a one-stop reference for developers and system admins who want to quickly get up to speed with using Hadoop v2. It would be helpful to have a basic knowledge of software development using Java and a basic working knowledge of Linux.