Installing a Storm Cluster

August 13, 2016 by S4

Filed under Hadoop, Strom

Last modified August 13, 2016

Installing a Storm Cluster

Following are the prerequisites for setting the cluster :

Linux Operating system
Java 6 installed
Python installed

Installation Steps :
Following steps are needed to get a Storm Cluster up and running.

Set up Zookeeper Cluster : Zookeeper is used as a coordinator in Storm cluster. You can refer here to see the installation steps for Zookeeper.
Install native dependencies on Nimbus & Worker machine There are some dependencies which are required by Storm i.e. ZeroMQ, JZMQ. These native dependencies are needed only on Storm cluster. While using Storm in local mode, Storm uses a pure Java Messaging system so you don’t need to install native dependencies there. But in cluster mode, it’s needed.

ZeroMQ 2.1.7 Installation : Storm has been tested with ZeroMQ 2.1.7, and this is the recommended ZeroMQ release that you install. You can download it from here. Following are the steps for installation.

wget http://download.zeromq.org/zeromq-2.1.7.tar.gz

tar –xzf zeromq-2.1.7.tar.gz

cd zeromq-2.1.7

./configure

make

sudo make install

JZMQ Installation: JZMQ is the java binding ZeroMQ. Here are the steps:

Download jzmq

cd jzmq

./autogen.sh

./configure

make

sudo make install

Download Storm release from here and copy it to every machine in cluster (nimbus and worker machine.
Configure storm.yaml : Storm release contains a file at the conf/storm.yaml with the default configuration to run the Storm Daemon.

storm.zookeeper.servers:
– “IP_MACHINE_1”
– “IP_MACHINE_2”
storm.local.dir: “/home/hadoop/STORM_DIR”
java.library.path: “/usr/local/lib”
nimbus.host: “IP_OF_NIMBUS_MACHINE”
supervisor.slots.ports:
– 6700
– 6701
– 6702
– 6703

Copy this storm.yaml into “~/.storm/” location.

Now launch all the Storm Daemons.

hduser@ubuntu:/usr/local/storm-0.8.1$ bin/storm nimbus &
hduser@ubuntu:/usr/local/storm-0.8.1$ bin/storm supervisor &
hduser@ubuntu:/usr/local/storm-0.8.1$ bin/storm ui &

Leave a Comment