Automating Hadoop Cluster Setup Using Ansible

Image for post
Image for post

What is Ansible?

What is Hadoop ?

Ansible Architecture :

Image for post
Image for post

Ansible Playbooks :

Inventory :

Control Node:

Managed Node:

Now lets start setup of Hadoop Cluster using Ansible.

Steps :

pip3 install ansible
yum install sshpass#To see version of ansible installed
ansible --version
Image for post
Image for post

Ansible Configuration File

vim /etc/ansible/ansible.conf
Image for post
Image for post

To check connectivity with all Managed Nodes

ansible all -m ping

PlayBook :

Image for post
Image for post
Image for post
Image for post
  • Running NameNode Playbook
ansible-playbook namenode.yml
Image for post
Image for post
  • Checking namenode services has been started in target node
Image for post
Image for post

PlayBook for Configration of DataNode

Image for post
Image for post
Image for post
Image for post
  • Running DataNode Playbook

(Due to low RAM and CPU resources I am making DataNode same VM that I used for NameNode. For that just stop the NameNode Services and delete the NameNode folder.)

ansible-playbook datanode.yml
Image for post
Image for post
Image for post
Image for post

PlayBook for Configration of Client

Image for post
Image for post

Link to Code :

Written by

I’m an undergraduate student at IIIT Ranchi, pursuing my B-Tech in Electronics and Communication Engineering.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store