Mining big data requires a deep investment in people and time. How can you be sure you’re building the right models? With this hands-on book, you’ll learn a flexible toolset and methodology for building effective analytics applications with Hadoop. Using lightweight tools such as Python, Apache Pig, and the D3.js library, your team will create an agile environment for exploring data, starting with an example application to mine your own email inboxes. You’ll learn an iterative approach that enables you to quickly change the kind of analysis you’re doing, depending on what the data is telling you. All example code in this book is available as working Heroku apps. Create analytics applications by using the agile big data development methodology Build value from your data in a series of agile sprints, using the data-value stack Gain insight by using several data structures to extract multiple features from a single dataset Visualize data with charts, and expose different aspects through interactive reports Use historical data to predict the future, and translate predictions into action Get feedback from users after each sprint to keep your project on track
Data science teams looking to turn research into useful analytics applications require not only the right tools, but also the right approach if they’re to succeed. With the revised second edition of this hands-on guide, up-and-coming data scientists will learn how to use the Agile Data Science development methodology to build data applications with Python, Apache Spark, Kafka, and other tools. Author Russell Jurney demonstrates how to compose a data platform for building, deploying, and refining analytics applications with Apache Kafka, MongoDB, ElasticSearch, d3.js, scikit-learn, and Apache Airflow. You’ll learn an iterative approach that lets you quickly change the kind of analysis you’re doing, depending on what the data is telling you. Publish data science work as a web application, and affect meaningful change in your organization. Build value from your data in a series of agile sprints, using the data-value pyramid Extract features for statistical models from a single dataset Visualize data with charts, and expose different aspects through interactive reports Use historical data to predict the future via classification and regression Translate predictions into actions Get feedback from users after each sprint to keep your project on track
Gain a practical introduction to DataOps, a new discipline for delivering data science at scale inspired by practices at companies such as Facebook, Uber, LinkedIn, Twitter, and eBay. Organizations need more than the latest AI algorithms, hottest tools, and best people to turn data into insight-driven action and useful analytical data products. Processes and thinking employed to manage and use data in the 20th century are a bottleneck for working effectively with the variety of data and advanced analytical use cases that organizations have today. This book provides the approach and methods to ensure continuous rapid use of data to create analytical data products and steer decision making. Practical DataOps shows you how to optimize the data supply chain from diverse raw data sources to the final data product, whether the goal is a machine learning model or other data-orientated output. The book provides an approach to eliminate wasted effort and improve collaboration between data producers, data consumers, and the rest of the organization through the adoption of lean thinking and agile software development principles. This book helps you to improve the speed and accuracy of analytical application development through data management and DevOps practices that securely expand data access, and rapidly increase the number of reproducible data products through automation, testing, and integration. The book also shows how to collect feedback and monitor performance to manage and continuously improve your processes and output. What You Will LearnDevelop a data strategy for your organization to help it reach its long-term goals Recognize and eliminate barriers to delivering data to users at scale Work on the right things for the right stakeholders through agile collaboration Create trust in data via rigorous testing and effective data management Build a culture of learning and continuous improvement through monitoring deployments and measuring outcomes Create cross-functional self-organizing teams focused on goals not reporting lines Build robust, trustworthy, data pipelines in support of AI, machine learning, and other analytical data products Who This Book Is For Data science and advanced analytics experts, CIOs, CDOs (chief data officers), chief analytics officers, business analysts, business team leaders, and IT professionals (data engineers, developers, architects, and DBAs) supporting data teams who want to dramatically increase the value their organization derives from data. The book is ideal for data professionals who want to overcome challenges of long delivery time, poor data quality, high maintenance costs, and scaling difficulties in getting data science output and machine learning into customer-facing production.
Using Agile methods, you can bring far greater innovation, value, and quality to any data warehousing (DW), business intelligence (BI), or analytics project. However, conventional Agile methods must be carefully adapted to address the unique characteristics of DW/BI projects. In Agile Analytics, Agile pioneer Ken Collier shows how to do just that. Collier introduces platform-agnostic Agile solutions for integrating infrastructures consisting of diverse operational, legacy, and specialty systems that mix commercial and custom code. Using working examples, he shows how to manage analytics development teams with widely diverse skill sets and how to support enormous and fast-growing data volumes. Collier's techniques offer optimal value whether your projects involve "back-end" data management, "front-end" business analysis, or both. Part I focuses on Agile project management techniques and delivery team coordination, introducing core practices that shape the way your Agile DW/BI project community can collaborate toward success Part II presents technical methods for enabling continuous delivery of business value at production-quality levels, including evolving superior designs; test-driven DW development; version control; and project automation Collier brings together proven solutions you can apply right now--whether you're an IT decision-maker, data warehouse professional, database administrator, business intelligence specialist, or database developer. With his help, you can mitigate project risk, improve business alignment, achieve better results--and have fun along the way.
You have to make sense of enormous amounts of data, and while the notion of "agile data warehousing might sound tricky, it can yield as much as a 3-to-1 speed advantage while cutting project costs in half. Bring this highly effective technique to your organization with the wisdom of agile data warehousing expert Ralph Hughes. Agile Data Warehousing Project Management will give you a thorough introduction to the method as you would practice it in the project room to build a serious "data mart. Regardless of where you are today, this step-by-step implementation guide will prepare you to join or even lead a team in visualizing, building, and validating a single component to an enterprise data warehouse. - Provides a thorough grounding on the mechanics of Scrum as well as practical advice on keeping your team on track - Includes strategies for getting accurate and actionable requirements from a team's business partner - Revolutionary estimating techniques that make forecasting labor far more understandable and accurate - Demonstrates a blends of Agile methods to simplify team management and synchronize inputs across IT specialties - Enables you and your teams to start simple and progress steadily to world-class performance levels
Agile Data Warehouse Design is a step-by-step guide for capturing data warehousing/business intelligence (DW/BI) requirements and turning them into high performance dimensional models in the most direct way: by modelstorming (data modeling + brainstorming) with BI stakeholders. This book describes BEAM✲, an agile approach to dimensional modeling, for improving communication between data warehouse designers, BI stakeholders and the whole DW/BI development team. BEAM✲ provides tools and techniques that will encourage DW/BI designers and developers to move away from their keyboards and entity relationship based tools and model interactively with their colleagues. The result is everyone thinks dimensionally from the outset! Developers understand how to efficiently implement dimensional modeling solutions. Business stakeholders feel ownership of the data warehouse they have created, and can already imagine how they will use it to answer their business questions. Within this book, you will learn: ✲ Agile dimensional modeling using Business Event Analysis & Modeling (BEAM✲) ✲ Modelstorming: data modeling that is quicker, more inclusive, more productive, and frankly more fun! ✲ Telling dimensional data stories using the 7Ws (who, what, when, where, how many, why and how) ✲ Modeling by example not abstraction; using data story themes, not crow's feet, to describe detail ✲ Storyboarding the data warehouse to discover conformed dimensions and plan iterative development ✲ Visual modeling: sketching timelines, charts and grids to model complex process measurement - simply ✲ Agile design documentation: enhancing star schemas with BEAM✲ dimensional shorthand notation ✲ Solving difficult DW/BI performance and usability problems with proven dimensional design patterns Lawrence Corr is a data warehouse designer and educator. As Principal of DecisionOne Consulting, he helps clients to review and simplify their data warehouse designs, and advises vendors on visual data modeling techniques. He regularly teaches agile dimensional modeling courses worldwide and has taught dimensional DW/BI skills to thousands of students. Jim Stagnitto is a data warehouse and master data management architect specializing in the healthcare, financial services, and information service industries. He is the founder of the data warehousing and data mining consulting firm Llumino.
Leverage DataRobot's enterprise AI platform and automated decision intelligence to extract business value from data Key FeaturesGet well-versed with DataRobot features using real-world examplesUse this all-in-one platform to build, monitor, and deploy ML models for handling the entire production life cycleMake use of advanced DataRobot capabilities to programmatically build and deploy a large number of ML modelsBook Description DataRobot enables data science teams to become more efficient and productive. This book helps you to address machine learning (ML) challenges with DataRobot's enterprise platform, enabling you to extract business value from data and rapidly create commercial impact for your organization. You'll begin by learning how to use DataRobot's features to perform data prep and cleansing tasks automatically. The book then covers best practices for building and deploying ML models, along with challenges faced while scaling them to handle complex business problems. Moving on, you'll perform exploratory data analysis (EDA) tasks to prepare your data to build ML models and ways to interpret results. You'll also discover how to analyze the model's predictions and turn them into actionable insights for business users. Next, you'll create model documentation for internal as well as compliance purposes and learn how the model gets deployed as an API. In addition, you'll find out how to operationalize and monitor the model's performance. Finally, you'll work with examples on time series forecasting, NLP, image processing, MLOps, and more using advanced DataRobot capabilities. By the end of this book, you'll have learned to use DataRobot's AutoML and MLOps features to scale ML model building by avoiding repetitive tasks and common errors. What you will learnUnderstand and solve business problems using DataRobotUse DataRobot to prepare your data and perform various data analysis tasks to start building modelsDevelop robust ML models and assess their results correctly before deploymentExplore various DataRobot functions and outputs to help you understand the models and select the one that best solves the business problemAnalyze a model's predictions and turn them into actionable insights for business usersUnderstand how DataRobot helps in governing, deploying, and maintaining ML modelsWho this book is for This book is for data scientists, data analysts, and data enthusiasts looking for a practical guide to building and deploying robust machine learning models using DataRobot. Experienced data scientists will also find this book helpful for rapidly exploring, building, and deploying a broader range of models. The book assumes a basic understanding of machine learning.
Build resilient applied machine learning teams that deliver better data products through adapting the guiding principles of the Agile Manifesto. Bringing together talented people to create a great applied machine learning team is no small feat. With developers and data scientists both contributing expertise in their respective fields, communication alone can be a challenge. Agile Machine Learning teaches you how to deliver superior data products through agile processes and to learn, by example, how to organize and manage a fast-paced team challenged with solving novel data problems at scale, in a production environment. The authors’ approach models the ground-breaking engineering principles described in the Agile Manifesto. The book provides further context, and contrasts the original principles with the requirements of systems that deliver a data product. What You'll Learn Effectively run a data engineering team that is metrics-focused, experiment-focused, and data-focused Make sound implementation and model exploration decisions based on the data and the metrics Know the importance of data wallowing: analyzing data in real time in a group setting Recognize the value of always being able to measure your current state objectively Understand data literacy, a key attribute of a reliable data engineer, from definitions to expectations Who This Book Is For Anyone who manages a machine learning team, or is responsible for creating production-ready inference components. Anyone responsible for data project workflow of sampling data; labeling, training, testing, improving, and maintaining models; and system and data metrics will also find this book useful. Readers should be familiar with software engineering and understand the basics of machine learning and working with data.
Contains a six-stage plan for starting new warehouse projects and guiding programmers step-by-step until they become a world-class, Agile development team. It describes also how to avoid or contain the fierce opposition that radically new methods can encounter from the traditionally-minded IS departments found in many large companies.
A guide to successfully operating in a lean-agile organization for solutions architects and enterprise architects Key FeaturesDevelop the right combination of processes and technical excellence to address architectural challengesExplore a range of architectural techniques to modernize legacy systemsDiscover how to design and continuously improve well-architected sustainable softwareBook Description Many organizations have embraced Agile methodologies to transform their ability to rapidly respond to constantly changing customer demands. However, in this melee, many enterprises often neglect to invest in architects by presuming architecture is not an intrinsic element of Agile software development. Since the role of an architect is not pre-defined in Agile, many organizations struggle to position architects, often resulting in friction with other roles or a failure to provide a clear learning path for architects to be productive. This book guides architects and organizations through new Agile ways of incrementally developing the architecture for delivering an uninterrupted, continuous flow of values that meets customer needs. You'll explore various aspects of Agile architecture and how it differs from traditional architecture. The book later covers Agile architects' responsibilities and how architects can add significant value by positioning themselves appropriately in the Agile flow of work. Through examples, you'll also learn concepts such as architectural decision backlog,the last responsible moment, value delivery, architecting for change, DevOps, and evolutionary collaboration. By the end of this Agile book, you'll be able to operate as an architect in Agile development initiatives and successfully architect reliable software systems. What you will learnAcquire clarity on the duties of architects in Agile developmentUnderstand architectural styles such as domain-driven design and microservicesIdentify the pitfalls of traditional architecture and learn how to develop solutionsUnderstand the principles of value and data-driven architectureDiscover DevOps and continuous delivery from an architect's perspectiveAdopt Lean-Agile documentation and governanceDevelop a set of personal and interpersonal qualitiesFind out how to lead the transformation to achieve organization-wide agilityWho this book is for This agile study guide is for architects currently working on agile development projects or aspiring to work on agile software delivery, irrespective of the methodology they are using. You will also find this book useful if you're a senior developer or a budding architect looking to understand an agile architect's role by embracing agile architecture strategies and a lean-agile mindset. To understand the concepts covered in this book easily, you need to have prior knowledge of basic agile development practices.
