EDG Testbed Install Recipe

Introduction

This is a quick and easy method for setting up a simple and working EDG testbed using LCFG including the initial LCFG server installation.

Presently this method can be used to install edg version 1.4.7.

Approximate times have been added to the major sections to give an idea of the time required.

LCFG Server

The LCFG server does really require a dedicated machine. It needs a minimum 10 Gigabyte hard disk but preferably a lot more. Other than that it can be a very plain machine.

This is very much based on the method presented by Julian Blake's recently updated cookbook method.

Install Redhat 6.2

Estimated time: 20 minutes + 40 minutes download time.

This is a standard redhat install using a CD or from the network (http,nfs and ftp). During the installtion the following should be observed.

Select a custom install.
Which ever partition contains /opt must contain space for at least 6 GBs, this is where the datagrid rpm repository will be located.
When choosing package groups the only requirements are Networked Machine and WWW/Mail/News Tools.
Finish of the installation.

Procede imediatley to the next section or switch your machine off as redhat 6.2 has a few security holes when first installed.

GPP Install Script

Estimated time: 20 minutes + 3 hours download time.

# lynx -source http://www.gridpp.ac.uk/gpp-install/v1_4_7_1/gpp-install.sh | sh

This script can be examined first.

To find out which release (presently, v1_4_7) you want to install check by asking someone on tb-support@jiscmail.ac.uk.

The script performs the following operations:

Hardens the machine slightly, switches off inetd, lpd , identd permanently. Switches off httpd, portmap and nfslock since they are possibly vuneralble at the moment. Also stops sendmail from listening.
Downloads the local rpm repository.
Configures /etc/ntp.conf and starts a network time protocol deamon.
Checksout the release of edg that you requested from Marianne's CVS.
Copies the release to the correct location.
Installs a newer glibc on the server using rpm.
Installs lots of rpms on the server using updaterpms.
Edits /etc/exports and /etc/services
Creates a symbolic link in your web servers document root so that the machine profiles are available on the web server.
Creates a symblic link to the nfsroot for the boot flopies.
Edits /etc/lilo.conf to reflect the new kernel that has been installed and runs /sbin/lilo.

This script aims to contain as much as possible of the configuration of LCFG that can be done automatically. The remaining parts of the configuration that have to be done require local configuration parameters.

After gpp-install

Estimated time: 5 minutes.

Check that lilo was able to install the MBR properly.
# /sbin/lilo
Added linux *

Reboot the machine.
shutdown -r now

You now should have a completley working LCFG server including everything you need to set up a EDG testbed.

Configure site-cfg.h

Estimated time: 1 hour.

The file /var/obj/conf/profile/source/site-cfg.h contains example configuration for your whole site. Copy site-cfg.h to site-cfg.h.ral You must configure the lines in this site-cfg.h.ral to reflect your local enviroment.

Here is a copy of the RAL site-cfg.h.ral for comparison.

The installation date must be a valid 'YYYYMMDDHHMMSSZ' the Z on the end denotes GMT.

I also recommend you use the following two values as well.

#define SITE_DEF_GRIDMAP    /share/grid-security/grid-mapfile
#define SITE_DEF_GRIDMAPDIR /share/grid-security/gridmapdir/

This is explained in the post lcfg notes.

The file contains comments about what all the values should be but again tb-support can help as well.

Configure Node Profiles

Estimated time: 40 minutes.

In /var/obj/conf/profile/source are four files testbed001-testbed004 that represent examples of CE, SE, UI and WN machines respectivley.

You should copy one of these , maybe testbed001 to gppce06 so you can create a CE. For the rest of this example I use gppce06 as my sample machine.

The file gppce06 needs editing:

Change the HOSTNAME to gppce06.
Edit the include so you include site-cfg.h.ral rather than the default site-cfg.h.legarno.
Append the line
+auth.rootpwd 56hk?8\$%=kjl
where you have previously generated a password crypt for root with.
# openssl passwd
Append the lines
+update.modlist eepro100
+update.mod_eepro100 alias eth0 e1000
This will add an entry in /etc/modules.conf for the node gppce06 so that when it reboots it has a chance of having a working network card. The choice if driver needs to reflect the actuall network card you have installed on your box, eg eepro, 3c509,...
Add #include "pbs-cfg.h" to the end of your CE's and WNs' configuration.
To the end of the WN's configuration add
#include "pbsexechost-cfg.h". To the end of the site-cfg.h file add
#define PBS_MASTER gppce06.gridpp.rl.ac.uk
where PBS_MASTER is usually your CE.

Common Grid Mapfile and Gridmapdir

To locate a common gridmap-file and gridmapdir in /share/grid-scecuiry across all the nodes:

To export from the CE edit '/var/obj/conf/profile/gppce06' and add. EXTRA(nfs.exports) gridsecurity nfs.fs_gridsecurity /share/grid-security nfs.options_gridsecurity \ SITE_SE_HOSTS_(rw,no_root_squash) SITE_WN_HOSTS(rw,no_root_squash) /* And Turn make grid-map on*/ +globus.servenabled_mkgmap yes The NFS exports allows the SE and WN to create entries in the gridmap directory. The second of these calls /sbin/chkconfig edg-gridmapfile-upgraded on.

On each of the WNs and SEs the directory must be mounted. On the end of their configuration files add. EXTRA(nfsmount.nfsmount) gridsecurity +nfsmount.nfsdetails_gridsecurity \ /share/grid-security CE_HOSTNAME:/share/grid-security rw
Finally the SE should not update its own grid-mapfile, to the end of SE's profile
DELETE(cron.additions,gmf)

Disable telnet,ftp and rlogin

By default the configuration enables telnet,ftp, rlogin on all the installed nodes. To the end of each node profile add.

DELETE(inet.services,telnet)
DELETE(inet.services,rlogin)
DELETE(inet.services,login)
DELETE(inet.services,ftp)

to disable each of these.

Compile Node Profiles

Estimated Time: 5 minutes.

Simply run the command.
# /usr/sbin/mkxprof -v -A gppce06.

After which you should be able to see the generated XML profile for that node at http://lcfgserver/profiles/gppce06.xml.

Internet Explorer is okay at browsing XML files.

Verify RPM Lists

Estimated Time: 2 minutes + 20 minutes execution time for each node type.

This test is now to strict and may well fail when in fact the installation will install.

For each of the node types there is a list rpms in /opt/local/linux/6.2/rpmcfg they are CE-rpm, WN-rpm, SE-rpm, NM-rpm,....

You should test the set of rpms you are about to install by issuing the following commands in
# gpp-testrpms.sh -l CE-rpm -r /tmp/testinstall

This will install all the rpms for a CE in /tmp/testinstall.

Any inconsistencies between the RPMS you have and the list you want to install will be flagged up at this point including rpm dependencies and conflicts.

LCFG Install Disk

Estimated Time: 10 minutes.

Download the LCFG install disk. Write it to a floppy.
# dd if=nfs_25032002.img of=/dev/fd0
Check the copy is good.
# cmp nfs_25032002.img /dev/fd0

DHCP

Estimated time: 20 minutes.

If you already have a DHCP server running on the same network segment as your testbed equipment and can reconfigure it, you should make the changes below on it. Otherwise, enable the DHCP server on the LCFG server machine with /sbin/chkconfig dhcpd on

Edit the /etc/dhcpd.conf file on the DHCP server add an entry for gppce06 so that when it boots it can obtain its network information.

deny unknown-clients;

subnet 130.246.184.0 netmask 255.255.252.0  {
     option domain-name "gridpp.rl.ac.uk";
     option time-servers 130.246.8.13;
     option routers 130.246.184.254;
     option domain-name-servers 130.246.8.13;

     host gppce06 {
       hardware ethernet 00:03:47:97:c9:f1;
       fixed-address 130.246.184.11;

       # Machine that contains the nfsroot.
       next-server lcfgserver.gridpp.rl.ac.uk;
       # Location of nfsroot.
       option root-path "/ir62";
       # Machine that contains XML profiles.
       next-server "gpplcfg.gridpp.rl.ac.uk";
       option option-151 "http://gpplcfg.gridpp.rl.ac.uk/";
    }
}

If you are using an existing DHCP server to provide ip addresses this is fine, just use the next-server and the option-151 to set the location of your LCFG server.

Now restart dhcpd.
/sbin/service dhcpd restart

If you are unsure of the mac address of your box. Start dhcpd, boot the machine and check the logs in /var/log/messages.

Install a Node

Estimated time: 2 minutes + 30 minutes install time. Put the floppy disk in the machine soon to be gppce06 and boot it up. Alternativly use PXE booting with some extra configuration.

Post Installation on Nodes

LCFG does do almost everything for you to configure the CE, SE, WN and UI. There are still a few steps to followed on each of the nodes.

Important Links

WP4 LCFG documentation.

EDG Installation Guide

Last modified Mon 21 April 2008 . View page history

Switch to HTTPS . Website Help . Print View . Built with GridSite 1.4.3

Sections

About GridPP

Documents

Activities

Management

Help

Information