Erlang Central

Difference between revisions of "ConnectingNodesByHand"

From ErlangCentral Wiki

(Typo!)
(4 intermediate revisions by 2 users not shown)
Line 1:Line 1:
If you wanted a simple 3 node cluster you could do the following by hand.
+
=== Problem ===
 +
With all of these fancy packages written by Erlang experts, how do you set up a cluster by hand?
  
#Note: On windows cygwin the shell works as written.
+
=== Motivation ===
#In win32 env's it's similar. except you should be creating .bat files instead of .sh files.
+
Show prospective users the basis for the Erlang magic instead of hiding it within the current set of  packages that are available.  But they are somewhat overwhelming for the novice user.
# On linux:
+
 
 +
== Solution ==
 +
If you wanted a simple three node cluster, you could execute the following steps by hand.
 +
 
 +
=== Prepare Some Utility Scripts ===
 +
'''Note:''' The following example is for Linux or other Unix variants (including Mac OS X).  On windows using cygwin the shell works as written. In a win32 environment, it's similar except you should be creating .bat files instead of .sh files.
 
# Install erlang somewhere, say /usr/local/otp_NNN
 
# Install erlang somewhere, say /usr/local/otp_NNN
 
# mkdir NodeStorage; mkdir NodeStorage/ANode; mkdir NodeStorage/BNode
 
# mkdir NodeStorage; mkdir NodeStorage/ANode; mkdir NodeStorage/BNode
Line 10:Line 16:
 
# mkdir NodeA NodeB NodeC
 
# mkdir NodeA NodeB NodeC
  
# The following script is useful if you have more than one box connected via NFS to the same filesystem.
+
The following script is useful if you have more than one box connected via NFS to the same filesystem. You will need to create a shell script (e.g., ''ANode.sh'') that contains the following:
# create a shell script (ANodeStart.sh) that contains the following:
+
##
+
 
     #!/bin/bash
 
     #!/bin/bash
 
 
     HOSTNAME=`hostname`
 
     HOSTNAME=`hostname`
 
     HDNAME=`host ${HOSTNAME} | cut -d ' ' -f 4`
 
     HDNAME=`host ${HOSTNAME} | cut -d ' ' -f 4`
 
     erl -name ANode@${HDNAME} -setcookie ACookie -mnesia dir '"~/NodeA"'
 
     erl -name ANode@${HDNAME} -setcookie ACookie -mnesia dir '"~/NodeA"'
  
# cp ANode.sh BNode.sh
+
Prepare the script for use in multiple sessions by executing the following commands in your shell (or console) of choice.  This assumes access to the UNIX '''sed''' command:
# sed -e 's/ANode/BNode/g'  ANode.sh > BNode.sh
+
    # cp ANode.sh BNode.sh
# sed -e 's/ANode/CNode/g'  ANode.sh > CNode.sh
+
    # sed -e 's/ANode/BNode/g'  ANode.sh > BNode.sh
 +
    # sed -e 's/ANode/CNode/g'  ANode.sh > CNode.sh
 +
 
 +
Each file should now contain something like:
 +
    erl (or erl.exe or werl.ex on windows) -name ANode@192.168.1.2 -setcookie ACookie -mnesia '"./NodeA"
 +
 
 +
=== Launch the Nodes ===
 +
In each of three separate console sessions, execute ''ANode.sh'', ''BNode.sh'', ''CNode.sh''
 +
 
 +
Each window should come up with an Erlang prompt:
 +
    1>
 +
 
 +
To get the '''nodename''', at the prompt, type:
 +
    1> node().
 +
    node@nohost
 +
 
 +
You can also quit at any time by typing:
 +
    2> q().
 +
    ok
 +
 
 +
=== Verify the Node Status ===
 +
For the communication between the nodes to work, they must share the same '''magic cookie''' values.  Verify this using the following command:
 +
    1> erlang:get_cookie().
 +
    nocookie
 +
    2>
 +
 
 +
If the cookies do not match, reset them by typing the following at the prompt (where '''SharedSecret''' is a value to be used in each node session):
 +
    2> erlang:set_cookie(node(), "SharedSecret").
 +
 
 +
Go to the ANode erlang shell, define two variables to represent the other nodes.  Note that the IP address values (192.168.1.2) should match the machine where BNode and CNode are running:
 +
    3> BNode='BNode@192.168.1.2'.
 +
    4> CNode='CNode@192.168.1.2'.
 +
 
 +
=== Use the Remote Nodes ===
 +
You can connect to other nodes from your current node:
 +
    5> net_adm:ping(BNode).
 +
    6> nodes().
 +
    7> net_adm:ping(CNode).
 +
    8> nodes().
 +
 
 +
Go to the other shells and verify that each node is connected to the other two nodes.  Note that by default it's a switch topology; other topologies are allowed.
 +
 
 +
Now that you are connected you can execute any loaded erlang module in any other node via rpc.
  
# Each file should now contain something like:
+
=== Examples ===
erl (or erl.exe or werl.ex on windows) -name ANode@192.168.1.2 -setcookie ACookie -mnesia '"./NodeA"
+
To find out the list of all files in "/tmp" on Node C from Node A.
 +
    9> rpc:call(CNode, file, list_dir, ["/tmp"]).
 +
To capture this into a variable:
 +
    10> {ok, CNodeUserLocalFileList} = rpc:call(CNode, file, list_dir, ["/usr/local"]).
 +
Or to capture this for all connected nodes:
 +
    11> rpc:multicall(nodes(), file, list_dir, ["/usr/local"]).
 +
Or to startup new processes on connected nodes
 +
    12> rpc:multicall(nodes(), os, cmd, ["/usr/local/tomcat/bin/startup.sh"]).
 +
To get an overview of your node cluster, use the process manager.
 +
    13> pman:start()
 +
    <0.31.0>
 +
This command brings up the process manager, where there is a Nodes sub menu which allows you to select which node's processes to inspect or trace; see the Erlang man page for details. Alternately, you can use the Erlang toolbar,
 +
    14> toolbar:start().
 +
    <0.37.0>
  
# Then in 3 separte windows, execute ANode.sh, BNode.sh, CNode.sh
 
# Each window should come up with an erlang prompt of #> .
 
# To quit the shell, at the prompt, type > q().
 
# To get nodename, at the prompt, type > node().
 
# To get the node cookie, at the prompt, type > erlang:get_cookie().
 
# Verify in each shell that the cookie is the same.
 
# If not reset the cookie, at the prompt, type > erlang:set_cookie(node(), "SharedSecret").
 
#
 
# Go to the ANode erlang shell, define 2 variables
 
> BNode='BNode@192.168.1.2'.
 
> CNode='CNode@192.168.1.2'.
 
# To Connect
 
> net_adm:ping(BNode).
 
> nodes().
 
> net_adm:ping(CNode).
 
> nodes().
 
# Go to the other shells and verify that each node is connected to the other 2 nodes.
 
# By default it's a switch topology;  other topologies are allowed.
 
# Now that you are connected you can execute any loaded erlang module in any other node via rpc.
 
# To find out the list of all files in "/tmp" on Node C from Node A.
 
> rpc:call(CNode, file, list_dir, ["/tmp"]).
 
# To capture this into a variable:
 
> {ok, CNodeUserLocalFileList} = rpc:call(CNode, file, list_dir, ["/usr/local"]).
 
# Or to capture this for all connected nodes:
 
> rpc:multicall(nodes(), file, list_dir, ["/usr/local"]).
 
# Or to startup new processes on connected nodes
 
> rpc:multicall(nodes(), os, cmd, ["/usr/local/tomcat/bin/startup.sh"]).
 
# To see what's going on in your node cluster, use
 
> pman:start()
 
# This brings up the process manager, there is a Nodes sub menu which allows you to select which node's processes to inspect or trace; see the erlang man page for details.
 
# Alternately, you can use the toolbar,
 
> toolbar:start().
 
 
[[Category:CookBook]]
 
[[Category:CookBook]]

Revision as of 06:07, 8 March 2007

Contents

Problem

With all of these fancy packages written by Erlang experts, how do you set up a cluster by hand?

Motivation

Show prospective users the basis for the Erlang magic instead of hiding it within the current set of packages that are available. But they are somewhat overwhelming for the novice user.

Solution

If you wanted a simple three node cluster, you could execute the following steps by hand.

Prepare Some Utility Scripts

Note: The following example is for Linux or other Unix variants (including Mac OS X). On windows using cygwin the shell works as written. In a win32 environment, it's similar except you should be creating .bat files instead of .sh files.

  1. Install erlang somewhere, say /usr/local/otp_NNN
  2. mkdir NodeStorage; mkdir NodeStorage/ANode; mkdir NodeStorage/BNode
  3. export ERL_TOP=(erlang install dir, in bash) or setenv ( csh )
  4. cd $ERL_TOP/bin
  5. mkdir NodeA NodeB NodeC

The following script is useful if you have more than one box connected via NFS to the same filesystem. You will need to create a shell script (e.g., ANode.sh) that contains the following:

    #!/bin/bash
    HOSTNAME=`hostname`
    HDNAME=`host ${HOSTNAME} | cut -d ' ' -f 4`
    erl -name ANode@${HDNAME} -setcookie ACookie -mnesia dir '"~/NodeA"'

Prepare the script for use in multiple sessions by executing the following commands in your shell (or console) of choice. This assumes access to the UNIX sed command:

    # cp ANode.sh BNode.sh
    # sed -e 's/ANode/BNode/g'  ANode.sh > BNode.sh
    # sed -e 's/ANode/CNode/g'  ANode.sh > CNode.sh

Each file should now contain something like:

    erl (or erl.exe or werl.ex on windows) -name ANode@192.168.1.2 -setcookie ACookie -mnesia '"./NodeA" 

Launch the Nodes

In each of three separate console sessions, execute ANode.sh, BNode.sh, CNode.sh

Each window should come up with an Erlang prompt:

    1>

To get the nodename, at the prompt, type:

    1> node().
    node@nohost

You can also quit at any time by typing:

    2> q().
    ok

Verify the Node Status

For the communication between the nodes to work, they must share the same magic cookie values. Verify this using the following command:

    1> erlang:get_cookie().
    nocookie
    2>

If the cookies do not match, reset them by typing the following at the prompt (where SharedSecret is a value to be used in each node session):

    2> erlang:set_cookie(node(), "SharedSecret").

Go to the ANode erlang shell, define two variables to represent the other nodes. Note that the IP address values (192.168.1.2) should match the machine where BNode and CNode are running:

    3> BNode='BNode@192.168.1.2'.
    4> CNode='CNode@192.168.1.2'.

Use the Remote Nodes

You can connect to other nodes from your current node:

    5> net_adm:ping(BNode).
    6> nodes().
    7> net_adm:ping(CNode).
    8> nodes().

Go to the other shells and verify that each node is connected to the other two nodes. Note that by default it's a switch topology; other topologies are allowed.

Now that you are connected you can execute any loaded erlang module in any other node via rpc.

Examples

To find out the list of all files in "/tmp" on Node C from Node A.

    9> rpc:call(CNode, file, list_dir, ["/tmp"]).

To capture this into a variable:

    10> {ok, CNodeUserLocalFileList} = rpc:call(CNode, file, list_dir, ["/usr/local"]).

Or to capture this for all connected nodes:

    11> rpc:multicall(nodes(), file, list_dir, ["/usr/local"]).

Or to startup new processes on connected nodes

    12> rpc:multicall(nodes(), os, cmd, ["/usr/local/tomcat/bin/startup.sh"]).

To get an overview of your node cluster, use the process manager.

    13> pman:start()
    <0.31.0>

This command brings up the process manager, where there is a Nodes sub menu which allows you to select which node's processes to inspect or trace; see the Erlang man page for details. Alternately, you can use the Erlang toolbar,

    14> toolbar:start().
    <0.37.0>