Apache Hadoop

  1. Hadoop Distributed File System (HDFS)
    1. HDFS Architecture
      1. Master-Slave Design
        1. Centralized Metadata Management
          1. Distributed Data Storage
          2. NameNode Components
            1. Namespace Management
              1. Block Location Tracking
                1. Metadata Operations
                  1. FsImage and EditLog
                    1. Checkpoint Process
                    2. DataNode Components
                      1. Block Storage Management
                        1. Data Serving Operations
                          1. Heartbeat Mechanism
                            1. Block Report Generation
                            2. Secondary NameNode
                              1. Checkpoint Creation
                                1. Metadata Backup
                                  1. Operational Differences from NameNode
                                2. HDFS Core Concepts
                                  1. Block Management
                                    1. Default Block Size
                                      1. Block Abstraction Benefits
                                        1. Block Placement Strategy
                                        2. Data Replication
                                          1. Replication Factor Configuration
                                            1. Rack Awareness
                                              1. Network Topology Considerations
                                                1. Replica Placement Policy
                                                2. Fault Tolerance Features
                                                  1. DataNode Failure Detection
                                                    1. Block Recovery Process
                                                      1. NameNode Failure Scenarios
                                                        1. Data Integrity Mechanisms
                                                        2. Data Integrity
                                                          1. Checksum Calculation
                                                            1. Corruption Detection
                                                              1. Recovery Procedures
                                                            2. HDFS Operations
                                                              1. Command Line Interface
                                                                1. Basic File Operations
                                                                  1. Directory Management
                                                                    1. Permission Management
                                                                      1. Administrative Commands
                                                                      2. Common HDFS Commands
                                                                        1. File Listing (ls)
                                                                          1. Directory Creation (mkdir)
                                                                            1. File Upload (put)
                                                                              1. File Download (get)
                                                                                1. File Viewing (cat)
                                                                                  1. File Deletion (rm)
                                                                                    1. Disk Usage (du)
                                                                                      1. Administrative Operations (dfsadmin)
                                                                                      2. HDFS Java API
                                                                                        1. FileSystem Class Usage
                                                                                          1. File Reading Operations
                                                                                            1. File Writing Operations
                                                                                              1. Exception Handling
                                                                                            2. Advanced HDFS Features
                                                                                              1. HDFS Federation
                                                                                                1. Multiple Namespace Support
                                                                                                  1. Namespace Isolation
                                                                                                    1. Scalability Benefits
                                                                                                    2. HDFS High Availability
                                                                                                      1. Active-Standby Configuration
                                                                                                        1. Automatic Failover
                                                                                                          1. Manual Failover
                                                                                                            1. Quorum Journal Manager
                                                                                                              1. Shared EditLog Management
                                                                                                                1. ZooKeeper Integration

                                                                                                            Previous

                                                                                                            2. Hadoop Architecture Overview

                                                                                                            Go to top

                                                                                                            Next

                                                                                                            4. Yet Another Resource Negotiator (YARN)