Oozie – Connection Refused Error on creating sharelib directory on hdfs

Please refer http://anggao.js.org/apache-oozie-installation-on-ubuntu.html

After preparing oozie  war file, we need to create sharelib directory on HDFS file system using the command  (we need to be in oozie home directory)

./bin/oozie-setup.sh sharelib create -fs hdfs://localhost:9000

The above command will internally issue a HDFS create directory command to the Name node running at hdfs://localhost:9000 and then copy the shared library to that directory.

Though we expect everything to be normal, sometimes errors may popup. The error I got was added below.


From the connection error shown on  screen, it is clear that name node at localhost:9000 is not  working. This can be confirmed by typing in terminal :

telnet localhost 9000 

Now I need to confirm the if that was the specified port for name node. This can be done using the command

# first get HDFS info
hdfs getconf -confKey fs.defaultFS

the above command gives the the output hdfs://localhost:9000.                                                              So we  are using correct path and port number

Now into the details of the issue.

The problem is name-node has not been started. So we need to ensure that all hadoop services are running.  Then we need to run the following command from hadoop user home (/home/hadoop for me) jps

This command is expected to give the following result if everything is normal,

2583 DataNode
2970 ResourceManager
3461 Jps
3177 NodeManager
2361 NameNode
2840 SecondaryNameNode
But when I executed the command, NameNode was missing from the result. That is the reason why we are not able to create sharelib on hdfs, Our real problem. So fixing this is the key to our problem.
This can be done as follows:
1. Move to hadoop user home (for me  /home/hadoop)
stop all hadoop services using  stop-all.sh
2. Then reformat name node using
   hdfs namenode -format
3. then restart hadoop all services using start-all.sh(or by specifically runningthe commands  start-dfs.sh and start-yarn.sh)
4.Then issue the command jps
5. If everything is normal will be getting earlier mentioned expected result. If that the case, we can resume with oozie installation. and run the command,
 ./bin/oozie-setup.sh sharelib create -fs hdfs://localhost:9000
But unfortunately, for me data node was missing in jps output. Fixing this issue is a bit tricky.
Before that stop all hadoop services using stop-all.sh
1. First we need to get the hdfs data node path. This will be specified in hdfs-site.xml of hadoop. (for me it was in /usr/local/hadoop/etc/hadoop)
2. for me data-node was in /home/hadoop/mydata/hdfs/datanode
3. cd to above path and remove all its contents by issuing rm command
   hadoop@space-Vostro-3800:/home/hadoop/mydata/hdfs/datanode$ rm -r *
4. Then go to hadoop user home folder and reformat name node .
   hadoop@space-Vostro-3800:/home/hadoop$ hdfs namenode -format
Then start all hadoop services using start-all.sh or by running the commands separately.
5. The run the command jps. Then I got the expected result
So here name node service is running and this can be verified by
                      telnet  localhost 9000
So everything is under control now . Now we can resume with oozie installation.  We can move to oozi folder in hadioop user home and run
hadoop@space-Vostro-3800:/home/hadoop/oozie$./bin/oozie-setup.sh sharelib create -fs hdfs://localhost:9000
Now we can create database using
hadoop@space-Vostro-3800:/home/hadoop/oozie$ ./bin/ooziedb.sh create -sqlfile oozie.sql -run
ok. with this database is created.
Now start oozie service using
hadoop@space-Vostro-3800:/home/hadoop/oozie$ ./bin/oozied.sh start
The same can be verified from web interface. Please type the following url in your browser.
# Web UI
Now we are done with so called oozie installation.




Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s