{"id":500,"date":"2011-02-15T12:57:53","date_gmt":"2011-02-15T17:57:53","guid":{"rendered":"http:\/\/hstore.cs.brown.edu\/"},"modified":"2012-04-14T10:13:07","modified_gmt":"2012-04-14T14:13:07","slug":"running-on-amazon-ec2","status":"publish","type":"page","link":"https:\/\/hstore.cs.brown.edu\/documentation\/deployment\/running-on-amazon-ec2\/","title":{"rendered":"Running on Amazon EC2"},"content":{"rendered":"

\u00ab<\/B> Transaction Trace Logs<\/a><\/div>
Automatic Deployment on Amazon EC2<\/a> \u00bb<\/B><\/div>
<\/div><\/p>\n

Note: These instructions are for manually setting up an H-Store cluster using NFS on Amazon EC2. See this page<\/a> for instructions on how to automatically deploy H-Store on EC2 using its built-in deployment scripts.<\/B><\/p>\n

The following instructions are how to setup a Ubuntu-based EC2 cluster to deploy H-Store, including how to use autofs to auto-mount the home directory through NFS on each node in the virtual cluster. This setup assumes that you will use the default ubuntu<\/tt> user for the Canonical-prepared EC2 images and that H-Store will be installed in \/home\/ubuntu\/H-Store<\/tt> ($HSTORE_HOME<\/tt>).<\/p>\n

    \n
  1. Create a new security group called hstore<\/i>. Allow for incoming SSH on port 22 so that you can log into each instance.<\/li>\n
  2. Launch a new EC2 Instance within the hstore<\/i> security group. In this example, we will use the official Ubuntu 10.04 64-bit community image in the US-East region (ami-4a0df923<\/a>).<\/li>\n
  3. Log into the instance as the ubuntu<\/tt> user. We need first to update the package repository list in order to get Sun’s version of Java (this is needed by Google Protocol Buffers). Add the two following lines to \/etc\/apt\/sources.list<\/tt>:\n
    \r\ndeb http:\/\/archive.canonical.com\/ubuntu lucid partner\r\ndeb-src http:\/\/archive.canonical.com\/ubuntu lucid partner\r\n<\/pre>\n
    Then install the required packages with the following command:<\/p>\n
    sudo apt-get update\r\nsudo apt-get --yes install subversion gcc g++ sun-java6-jdk valgrind ant<\/pre>\n<\/li>\n
  4.  Check out the source for H-Store from Subversion. For this example, we will use the trunk:\n
    \r\nsvn co https:\/\/database.cs.brown.edu\/svn\/hstore\/trunk\/<\/a> $HSTORE_HOME\r\n<\/pre>\n<\/li>\n
  5. Copy your key pair file generated by Amazon into the $HOME\/.ssh\/<\/tt> directory.\n
    cp hstore.pem ~\/.ssh\/ && chmod 0400  ~\/.ssh\/hstore.pem<\/pre>\n<\/li>\n
     Change the global.sshoptions<\/a> parameter in $HSTORE_HOME\/properties\/default.properties<\/tt> to be the following:<\/p>\n
    global.sshoptions = -i \/home\/ubuntu\/.ssh\/hstore.pem<\/pre>\n<\/li>\n
  6. Now build the project:\n
    cd $HSTORE_HOME && ant build<\/tt><\/pre>\n<\/li>\n
     You can test that the system is working properly on a single node using the following command:<\/p>\n
    ant hstore-prepare<\/tt> hstore-benchmark<\/tt> -Dproject=tpcc<\/pre>\n<\/li>\n
  7. Although not required, it is easier to run H-Store when using a networked file system. The following are instructions on how to create a NFS server using a micro instance of Ubuntu. We will export the directory $HSTORE_HOME<\/tt> and use autofs<\/a>  to automatically mount them on each machine in the virtual cluster.\n
      \n
    1. Create the instance:\n
      ec2-run-instances --region us-east-1 \\\r\n                  --group hstore \\\r\n                  -k hstore \\\r\n                  --instance-type t1.micro ami-4a0df923<\/a><\/pre>\n
      You can also set associate instance to an elastic IP address to ensure that the nodes can always connect to it even after it is restarted. Then add an entry to \/etc\/hosts<\/tt> that maps the hostname hstore-nfs<\/tt> to this IP address. This is the hostname we will use in the examples below.\n<\/li>\n
    2. Log into server and execute the following commands as root to setup NFS. Note that these steps are based on the instructions found here<\/a>.\n
      \r\napt-get install nfs-kernel-server \r\necho \"\/home *(rw,async,no_subtree_check)\" >> \/etc\/exports\r\nexportfs  -a\r\nservice portmap start\r\n\/etc\/init.d\/nfs-kernel-server start\r\n<\/pre>\n<\/li>\n
    3. Enable all internal network traffic for the hstore<\/tt> security group.<\/li>\n
    4. We now need to configure autofs on each of the client machines. You can do this step once, and then clone the instance. First, install the autofs package:\n
      apt-get install autofs<\/pre>\n<\/li>\n
    5. If you are using the hostname alias hstore-nfs<\/tt> mentioned above, edit the \/etc\/hosts<\/tt> file and add in an entry at the end of the file. Replace the IP address shown below with the private IP address<\/a> for the NFS head node instance. Do not use the Elastic IP address because it will not be able to go through the firewall.\n
      10.XXX.XXX.XXX cs227-nfs<\/pre>\n<\/li>\n
    6. You can test that the configuration is working by using the mount command directly:\n
      mount -t nfs hstore-nfs:\/home\/ubuntu\/H-Store \/home\/ubuntu\/H-Store<\/pre>\n<\/li>\n
    7. Now edit the \/etc\/auto.master<\/tt> file and add in the following line at the bottom of the file\n
      \/home         \/etc\/auto.home<\/pre>\n<\/li>\n
    8. Create a new file \/etc\/auto.home<\/tt> and add in the following information. Note that the directory needs to end with a ‘\/<\/tt>‘ followed by a ‘&<\/tt>‘\n
      *    hstore-nfs:\/home\/&<\/pre>\n<\/li>\n
    9. Lastly, start the autofs service\n
      \/etc\/init.d\/autofs start<\/pre>\n<\/li>\n<\/li>\n<\/ol>\n
    10. Once NFS is installed on each of the nodes in the cluster and you have installed the source code in an NFS shared directory, you can create a cluster configuration file and update the benchmark catalog. H-Store will automatically log into each node and launch the HStoreSite<\/tt> service off of the directory.<\/li>\n<\/ol>\n
      \u00ab<\/B> Transaction Trace Logs<\/a><\/div> 
      Automatic Deployment on Amazon EC2<\/a> \u00bb<\/B><\/div>
      <\/div><\/p>\n","protected":false},"excerpt":{"rendered":"
      Note: These instructions are for manually setting up an H-Store cluster using NFS on Amazon EC2. See this page for instructions on how to automatically deploy H-Store on EC2 using its built-in deployment scripts. The following instructions are how to setup a Ubuntu-based EC2 cluster to deploy H-Store, including how to use autofs to auto-mount […]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":626,"menu_order":98,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/pages\/500"}],"collection":[{"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/comments?post=500"}],"version-history":[{"count":48,"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/pages\/500\/revisions"}],"predecessor-version":[{"id":1597,"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/pages\/500\/revisions\/1597"}],"up":[{"embeddable":true,"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/pages\/626"}],"wp:attachment":[{"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/media?parent=500"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}