Cloudera - Certified Developer for Apache Hadoop (CCDH)

Lehrplan

The Motivation for Hadoop

• Problems with traditional large-scale systems
• Introducing Hadoop
• Hadoopable problems

Hadoop: Basic Concepts and HDFS

• The Hadoop project and Hadoop components
• The Hadoop Distributed File System

Introduction to MapReduce

• MapReduce overview
• Example: WordCount
• Mappers
• Reducers

Hadoop Clusters and the Hadoop Ecosystem

• Hadoop cluster overview
• Hadoop jobs and tasks
• Other Hadoop ecosystem components

Writing a MapReduce Program in Java

• Basic MapReduce API Concepts
• Writing MapReduce Drivers, Mappers, and Reducers in Java
• Speeding up Hadoop development by using eclipse
• Differences between the old and new MapReduce APIs

Writing a MapReduce Program Using Streaming

• Writing Mappers and Reducers with the streaming API

Unit Testing MapReduce Programs

• Unit testing
• The JUnit and MRUnit testing frameworks
• Writing unit tests with MRUnit
• Running unit tests

Delving Deeper into the Hadoop API

• Using the ToolRunner class
• Setting up and tearing down Mappers and Reducers
• Decreasing the amount of intermediate data with combiners
• Accessing HDFS programmatically
• Using the distributed cache
• Using the Hadoop API’s Library of Mappers, Reducers, and Partitioners

Practical Development Tips and Techniques

• Strategies for debugging MapReduce code
• Testing MapReduce code locally by using LocalJobRunner
• Writing and viewing log files
• Retrieving job information with counters
• Reusing objects
• Creating map-only MapReduce jobs

Partitioners and Reducers

• How partitioners and Reducers work together
• Determining the optimal number of Reducers for a job
• Writing customer partitioners

Data Input and Output

• Creating custom writable and WritableComparable implementations
• Saving binary data using sequenceFile and Avro data files
• Issues to consider when using file compression
• Implementing custom InputFormats and OutputFormats

Common MapReduce Algorithms

• Sorting and searching large data sets
• Indexing data
• Computing term frequency — Inverse Document Frequency
• Calculating word co-occurrence
• Performing Secondary Sort

Joining Data Sets in MapReduce Jobs

• Writing a Map-Side Join
• Writing a Reduce-Side Join

Integrating Hadoop into the Enterprise Workflow

• Integrating Hadoop into an existing enterprise
• Loading data from an RDBMS into HDFS by using Sqoop
• Managing real-time data using Flume
• Accessing HDFS from legacy systems with FuseDFS and HttpFS

An Introduction to Hive, Imapala, and Pig

• The motivation for Hive, Impala, and Pig
• Hive overview
• Impala overview
• Pig overview
• Choosing Between Hive, Impala, and Pig

An Introduction to Oozie

• Introduction to Oozie
• Creating Oozie workflows