cbaWorkflow : Hadoop cluster hardware setup

Cluster configuration:

2 namenodes(active and standby)

5 datanodes

2 edge nodes(dev and prod)


Hosts

hostOSIPrackposition
r02edgeCentOS Linux release 7.8.2003 (Core)192.168.2.3022
r01edgeCentOS Linux release 7.8.2003 (Core)192.168.2.2225
datanode1CentOS Linux release 7.8.2003 (Core)192.168.2.2311
datanode2CentOS Linux release 7.8.2003 (Core)192.168.2.2421
datanode3CentOS Linux release 7.8.2003 (Core)192.168.2.2723
datanode4CentOS Linux release 7.8.2003 (Core)192.168.2.2813
datanode5CentOS Linux release 7.8.2003 (Core)192.168.2.2912
namenode1CentOS Linux release 7.8.2003 (Core)192.168.2.2514
namenode2CentOS Linux release 7.8.2003 (Core)192.168.2.2624

OS: RHEL7 has been replaced by CentOS Linux release 7.8.2003 (Core) – no yum update is possible in RHEL 7 without subscription

JIRA:HTOD-41

Hardware diagram

Hardware setup

  1. DELL PowerEdge R710 and DELL PowerEdge R610
  2. RAM as per supported configuration
  3. Harddrives:

    Namenodes: RAID1 for system,  RAID0 for data(fsimage and edits)

    Datanodes: RAID0

  1. Network adapters -4 each Adding bonding for multiple network adapters

Routers:

2 Netgear 24 port switches(1G),interconnected

(optional 1 optical 10G,connecting 2 netgear switches


Rack, power outlets, UPS, Ventilation.

Creating network cables of proper size.

Assembling and mounting the computers, connecting network

Labeling the computers with the system specifications and storing the inventory

Benchmark the disk speed

yum install hdparm

[root@datanode4 ~]#hdparm -Tt /dev/sdb

/dev/sdb:

Timing cached reads: 15404 MB in 1.99 seconds = 7738.06 MB/sec
Timing buffered disk reads: 1344 MB in 3.00 seconds = 447.97 MB/sec


[root@datanode4 ~]# hdparm -Tt /dev/sdc

/dev/sdc:
Timing cached reads: 14834 MB in 1.99 seconds = 7450.71 MB/sec
Timing buffered disk reads: 934 MB in 3.00 seconds = 311.31 MB/sec

Attachments:

Hardware.png (image/png)