![\"\"](\"https:\/\/techieshouts.com\/wp-content\/uploads\/2019\/07\/HadoopInstallation-1.jpg\")
<\/figure>In realtime, Hadoop<\/a> will be installed into a network of machines to form a cluster. Here, in this article, we will see the installation of Hadoop step by step in a single Ubuntu system. The post is written with an assumption that you already know what is a Name node, Data node, HDFS<\/a>, etc and the basic Hadoop<\/a> components. If not, please visit the previous articles to know about these basic concepts.<\/p>\n<\/div><\/div>\n\n\n\n <\/p>\n\n\n\n Open the core-site.xml in the path \u201c\/home\/user\/hadoop-2.7.1\/etc\/hadoop\u201d using a text editor. Here you need to set the default file system and temp directory.<\/p>\n\n\n\n Here you need to set the path for name node and data node.<\/p>\n\n\n\n Here you will set the properties related to node manager and resource manager(MR2)<\/p>\n\n\n\n You will see mapred-site.xml.template in the same folder. Just create a copy of that and rename it to mapred-site.xml. Here you can set the properties about MR framework and JobHistory web page<\/p>\n\n\n\n After setting up the above configurations, you need to add localhost to the slaves file. This is because we are setting up the pseudo-cluster which means, both master and slave will be localhost.<\/p>\n\n\n\n Navigate to ~hadoop-2.7.1\/etc\/hadoop path and open the “slaves” file in editor. Just add “localhost” to the file and save it.<\/p>\n\n\n\n All set for launching Hadoop in your machine. Now you can start and stop Hadoop daemons using the following commands.<\/p>\n\n\n\n start-all.sh \u2013 <\/strong>Five daemons of Hadoop: Namenodes, data nodes, secondary name node, resource manager, node manager will be started.<\/p>\n\n\n\n start-dfs.sh<\/strong> \u2013 The first 3 daemons will be started from the above list.<\/p>\n\n\n\n start-yarn.sh \u2013 <\/strong>The last 2 daemons(Yarn daemons) will be started.<\/p>\n\n\n\n You can also start the specific daemon by running<\/p>\n\n\n\n hadoop-daemons.sh start namenode<\/strong><\/p>\n\n\n\n To stop the daemons use the below commands<\/p>\n\n\n\n stop-all.sh<\/strong> \u2013 Will stop all daemons<\/p>\n\n\n\n stop-dfs.sh<\/strong> \u2013 Will stop the dfs daemons.<\/p>\n\n\n\n <\/p>\n","protected":false},"excerpt":{"rendered":" In realtime, Hadoop will be installed into a network of machines to form a cluster. Here, in this article, we will see the installation of Hadoop step by step in a single Ubuntu system. The post is written with an assumption that you already know what is a Name node, Data node, HDFS, etc and\u2026 Read More »<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":304,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[11,125],"tags":[25,65,22,64],"_links":{"self":[{"href":"https:\/\/techieshouts.com\/home\/wp-json\/wp\/v2\/posts\/292"}],"collection":[{"href":"https:\/\/techieshouts.com\/home\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techieshouts.com\/home\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techieshouts.com\/home\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/techieshouts.com\/home\/wp-json\/wp\/v2\/comments?post=292"}],"version-history":[{"count":16,"href":"https:\/\/techieshouts.com\/home\/wp-json\/wp\/v2\/posts\/292\/revisions"}],"predecessor-version":[{"id":326,"href":"https:\/\/techieshouts.com\/home\/wp-json\/wp\/v2\/posts\/292\/revisions\/326"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techieshouts.com\/home\/wp-json\/wp\/v2\/media\/304"}],"wp:attachment":[{"href":"https:\/\/techieshouts.com\/home\/wp-json\/wp\/v2\/media?parent=292"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techieshouts.com\/home\/wp-json\/wp\/v2\/categories?post=292"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techieshouts.com\/home\/wp-json\/wp\/v2\/tags?post=292"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
export HADOOP_PREFIX=\/home\/user\/hadoop-2.7.1
export HADOOP_HOME=${HADOOP_PREFIX}
export HADOOP_CONF_DIR=${HADOOP_PREFIX}\/etc\/hadoop
export PATH=$JAVA_HOME\/bin:$HADOOP_HOME\/bin:$HADOOP_HOME\/sbin:$PATH <\/li> Setting up core-site.xml<\/strong><\/h2>\n\n\n\n
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<configuration>\n <property>\n <name>fs.defaultFS<\/name>\n <value>hdfs:\/\/localhost:8020<\/value>\n <\/property>\n <property>\n <name>hadoop.tmp.dir<\/name>\n <value>\/home\/user\/tmp<\/value>\n <\/property>\n<\/configuration><\/pre>\n\n\n\n
Setting up hdfs-site.xml<\/strong><\/h2>\n\n\n\n
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<configuration>\n <property>\n <name>dfs.namenode.name.dir<\/name>\n <value>\/home\/user\/name<\/value>\n <\/property>\n <property>\n <name>dfs.datanode.data.dir<\/name>\n <value>\/home\/user\/data<\/value>\n <\/property>\n<\/configuration><\/pre>\n\n\n\n
Setting up yarn-site.xml<\/strong><\/h2>\n\n\n\n
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<configuration>\n <property>\n <name>yarn.resourcemanager.hostname<\/name>\n <value>localhost<\/value>\n <\/property>\n <property>\n <name>yarn.nodemanager.aux-services<\/name>\n <value>mapreduce_shuffle<\/value>\n <\/property>\n <property>\n <name>yarn.log-aggregation-enable<\/name>\n <value>true<\/value>\n <\/property>\n <property>\n <name>yarn.nodemanager.remote-app-log-dir<\/name>\n <value>hdfs:\/\/localhost:8020\/log\/<\/value>\n <\/property>\n<\/configuration><\/pre>\n\n\n\n
Setting up mapred-site.xml<\/strong><\/h2>\n\n\n\n
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<configuration>\n <property>\n <name>mapreduce.framework.name<\/name>\n <value>yarn<\/value>\n <\/property>\n <property>\n <name>mapreduce.jobhistory.address<\/name>\n <value>localhost:10020<\/value>\n <\/property>\n <property>\n <name>mapreduce.jobhistory.webapp.address<\/name>\n <value>localhost:19888<\/value>\n <\/property>\n<\/configuration><\/pre>\n\n\n\n
Updating slaves file<\/h2>\n\n\n\n
cd $HADOOP_CONF_DIR\nhdfs namenode -format<\/pre>\n\n\n\n
~$ sudo apt-get install openssh-server\n~$ sudo service ssh start\n~$ ssh-keygen\n~$ cd .ssh\n~$ cat id_rsa.pub >> authorized_keys\n~$ chmod 600 authorized_keys<\/pre>\n\n\n\n