Glasgow Job Wrapper Worked Example (Deprecated)
Contents
Running executables with GANGA
First of all log in and initialise your proxy as normal:
- Login to ppeui.
- Initialise a proxy
$ grid-proxy-init
Copying Necessary Files
- Now copy over all the files necessary to run your job using gsisftp
$ gsisftp -oPort=2222 svr020.gla.scotgrid.ac.uk
This will open up sftp, in this mode you can copy over files using 'put'
sftp> put test.exe Uploading test.exe to /clusterhome/home/gla019/test.exe test.exe
Remember to replace test.exe with your own files and gla019 with your username. Exit sftp using CTRL-D.
Now login to svr020.gla.scotgrid.ac.uk on port 222 using
$ gsissh -p 2222 svr020.gla.scotgrid.ac.uk
and copy the file you transfered to your shared directory
$ cp test.exe /cluster/share/gla019/
Running Ganga
If you are new to Ganga familiarise yourself with the Glasgow_Ganga_Quickstart_Guide. When ready activate Ganga as normal:
$ganga
To run on the grid use
$ In [4]:gridJob=Job(backend=LCG(),application=Executable(exe=File('/cluster/share/gla019/test.exe'), args=['1','2','3']))
This is equivalent to executing the command
$/cluster/share/gla019/test.exe 1 2 3
and will run the job anywhere your VO is supported. However you can target specific sites if you wish, for instance to target glasgow you would use:
$ In [5]:gridJob.backend.CE='svr016.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-dteam'
if your VO is dteam. To actually submit the job use
In [6]:gridJob.submit() Ganga.GPIDev.Lib.Job : INFO submitting job 3 Ganga.GPIDev.Adapters : INFO submitting job 3 to LCG backend Ganga.GPIDev.Lib.Job : INFO job 3 status changed to "submitted" Out[6]: 1
Checking Job Output
If you want to check the status of your job use the Ganga command:
$ In [3]: gridJob
If you want book keeping information simply look for the id string which takes the form:
$ https://svr023.gla.scotgrid.ac.uk:9000/T2tfzxgeX6JXxq86KgSA
Then exit ganga and run
$ edg-job-status https://svr023.gla.scotgrid.ac.uk:9000/zFIpb-Y1BgiU_EEKM9Hp9g ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://svr023.gla.scotgrid.ac.uk:9000/zFIpb-Y1BgiU_EEKM9Hp9g Current Status: Scheduled Status Reason: Job successfully submitted to Globus Destination: e5grid05.physik.uni-dortmund.de:2119/jobmanager-lcgpbs-dteam reached on: Wed Jun 13 13:40:21 2007 *************************************************************
This will return job information information, in this case explaining that your job is scheduled to run. Once the job is completed the query will return
-bash-2.05b$ edg-job-status https://svr023.gla.scotgrid.ac.uk:9000/zFIpb-Y1BgiU_EEKM9Hp9g ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : https://svr023.gla.scotgrid.ac.uk:9000/zFIpb-Y1BgiU_EEKM9Hp9g Current Status: Cleared Status Reason: user retrieved output sandbox Destination: e5grid05.physik.uni-dortmund.de:2119/jobmanager-lcgpbs-dteam reached on: Wed Jun 13 14:37:24 2007 *************************************************************
And you can collect your output.
You can also check on the output in Ganga once the job is completed:
In [7]:!cat $gridJob.outputdir/stdout
The exclamation mark before the cat command above is the way to tell IPYTHON that you issue a shell command. The dollar sign indicates that you return to a python command. By default, ganga will store jobs' outputs in ~/gangadir/workspace/Local/JOB_ID/output, where JOB_ID is a sequential job number.
This mechanism is called an input sandbox and is designed to handle small files (up to few MB). More information about handling files can be found in the ganga documentation.
Running Multiple jobs with Ganga
If you wish to run multiple jobs on the Grid with Ganga you can use a simple loop structure. The structure for submitting 100 jobs is given here:
In [24]:a=list() In [25]:for i in range(100): ....: a.append(Executable(exe='/bin/echo', args=[str(i)])) ....: In [26]:s=ExeSplitter(apps=a) In [27]:j=Job(splitter=s,backend=LCG()) In [28]:j.submit()