Grid

Building your own grid

A key idea of GridFactory is to allow anyone to create her own computing grid - and connect it to other grids. Notice however that:

  • To share computing resources with others you don’t have to create your own grid. You can simply install GridWorker, connect to an existing GridFactory server and run jobs for an existing virtual organization of your own choice (see share).
  • To run computations on remote resources you also don’t have to create your own grid. You can simply install GridPilot, join a virtual organization and submit jobs to an existing GridFactory server (see compute).

I want to create my own grid. How do I proceed?

factory_small

You have two options:
  • Use hosted services: see services.
  • Set up your own GridFactory server, following the instructions below.

Server installation instructions


– Notice that currently the software has only been tested on Ubuntu-9.10, 10.04 and Fedora-12, 13 - i386*
– Notice also that you must first install SUN/Oracle’s Java (>=1.6), either from a distribution repository or directly from Oracle.

Download and installation on Ubuntu

Download
Download mod_gacl, mod_gridfactory, gridfactory_server: either use your browser and get the files from the download page, or use e.g. wget:

wget http://www.gridfactory.org/downloads/gridfactory/gridfactory_server/mod-gacl_0.9.1-2_i386.deb
wget http://www.gridfactory.org/downloads/gridfactory/gridfactory_server/mod-gridfactory_0.1.6-2_i386.deb
wget http://www.gridfactory.org/downloads/gridfactory/gridfactory_server/gridfactory-server_0.1.5-2_i386.deb
wget http://www.gridfactory.org/downloads/gridfactory/gridfactory_command_line/gridfactory-cli_0.1.5-2_i386.deb
Installation - as root
For 10.04, first do:
apt-get install python-software-properties 
add-apt-repository "deb http://archive.canonical.com/ lucid partner"
apt-get update
For all supported Ubuntu versions: 
apt-get install curl apache2 mysql-server libaprutil1-dbd-mysql sun-java6-jre sysv-rc-conf

dpkg --force-overwrite -i *.deb

Download and installation on Fedora

Download
Download mod_gacl, mod_gridfactory, gridfactory_server: either use your browser and get the files from the download page, or use e.g. wget:

wget http://www.gridfactory.org/downloads/gridfactory/gridfactory_server/mod_gacl_linux_fedora_0_9_1.rpm
wget http://www.gridfactory.org/downloads/gridfactory/gridfactory_server/mod_gridfactory_linux_fedora_0_1_6.rpm
wget http://www.gridfactory.org/downloads/gridfactory/gridfactory_server/gridfactory_server_linux_0_1_5.rpm
wget http://www.gridfactory.org/downloads/gridfactory/gridfactory_command_line/gridfactory_cli_linux_0_1_5.rpm
Installation - as root
yum install curl httpd mod_ssl mysql-server apr-util-mysql

rpm -Uvh --force *.rpm

/usr/sbin/alternatives --install /usr/bin/java java /usr/java/default/bin/java 20000
/usr/sbin/setenforce 0
mkdir /etc/httpd/conf.d.ina
mv /etc/httpd/conf.d/*.conf /etc/httpd/conf.d.ina/
/etc/init.d/mysqld start
mysqladmin -u root password my_password

Configuration - as root

  • make sure the MySQL is running and you know administrator user name and password
  • optionally (but recommended) put a server SSL certificate + key (unencrypted) in /var/www/grid/hostcert.pem and /var/www/grid/hostkey.pem respectively
  • run the configuration script:
    /usr/share/gridfactory/configure_services.sh

Checks - as a normal user

  • import the certificate /usr/share/gridfactory/cli/testcertkey.p12 into your browser (password “dummy”)
  • Check that https://server_host_name/gridfactory/ can be read
  • Check that https://server_host_name/db/jobs/ can be read
  • Set up CA certificates:
    mkdir certificates
cp /var/www/grid/certificates/CABO.pem certificates/`openssl x509 -in /var/www/grid/certificates/CABO.pem -hash | head -1`.0
    • Ubuntu:
      cp /etc/ssl/certs/ssl-cert-snakeoil.pem certificates/`openssl x509 -in /etc/ssl/certs/ssl-cert-snakeoil.pem -hash | head -1`.0
    • Fedora:
      sudo cp /etc/pki/tls/certs/localhost.crt /tmp/
sudo chmod go+r /tmp/localhost.crt
cp /tmp/localhost.crt certificates/`openssl x509 -in /tmp/localhost.crt -hash | head -1`.0
sudo rm /tmp/localhost.crt
  • Submit a simple test job:
    echo "echo hello" > hello.sh
/usr/share/gridfactory/cli/psub -a certificates -b server_host_name hello.sh
/usr/share/gridfactory/cli/pstat -a certificates -b server_host_name
/usr/share/gridfactory/cli/pclean -a certificates -b server_host_name
To actually have this job executed, you’ll have to one or several GridWorkers that pull jobs from your server. Once you have that in place, you may want to run some more advanced example jobs.

NOTICE: Don’t use localhost as server_host_name. If you don’t have network, use localhost.localdomain or similar instead. Using localhost triggers direct write to the file system, for which you must set permissions appropriately.

You should also have a look at the documentation page and configure your services and clients further. In particular you should get or create your own user and server certificates and modify your virtual organization, https://server_host_name/vos/default.txt, by editing /var/www/grid/data/vos/default.txt.

(*) The main issue with other/older distros is that the Apache modules depend on Apache >=2.1 and libaprutil-1 with mysql support (check e.g. with ls /usr/lib/apr-util-1/apr_dbd_mysql-1.so or nm /usr/lib/libaprutil-1.so.0 | grep mysql). In the case of ubuntu 9.10, you can simply execute apt-get install libaprutil1-dbd-mysql. Binaries should exist for other distributions too, but for older distributions you may have to compile yourself (see the README file for details).

Comments (1) ยป