Glasgow Job Wrapper Worked Example (Deprecated)

Running executables with GANGA

First of all log in and initialise your proxy as normal:

• Login to ppeui.
• Initialise a proxy

 $ grid-proxy-init

Copying Necessary Files

• Now copy over all the files necessary to run your job using gsisftp

 $ gsisftp -oPort=2222  svr020.gla.scotgrid.ac.uk

This will open up sftp, in this mode you can copy over files using 'put'

  sftp> put test.exe
  Uploading test.exe to /clusterhome/home/gla019/test.exe
  test.exe 

Remember to replace test.exe with your own files and gla019 with your username. Exit sftp using CTRL-D.

Now login to svr020.gla.scotgrid.ac.uk on port 222 using

 $ gsissh -p 2222 svr020.gla.scotgrid.ac.uk

and copy the file you transfered to your shared directory

 $ cp test.exe /cluster/share/gla019/

Running Ganga

If you are new to Ganga familiarise yourself with the Glasgow_Ganga_Quickstart_Guide. When ready activate Ganga as normal:

    $ganga

To run on the grid use

     $ In [4]:gridJob=Job(backend=LCG(),application=Executable(exe=File('/cluster/share/gla019/test.exe'), 
args=['1','2','3']))

This is equivalent to executing the command

   $/cluster/share/gla019/test.exe 1 2 3

and will run the job anywhere your VO is supported. However you can target specific sites if you wish, for instance to target glasgow you would use:

  $ In [5]:gridJob.backend.CE='svr016.gla.scotgrid.ac.uk:2119/jobmanager-lcgpbs-dteam'

if your VO is dteam. To actually submit the job use

In [6]:gridJob.submit()
Ganga.GPIDev.Lib.Job               : INFO     submitting job 3
Ganga.GPIDev.Adapters              : INFO     submitting job 3 to LCG backend
Ganga.GPIDev.Lib.Job               : INFO     job 3 status changed to "submitted"
Out[6]: 1

Checking Job Output

If you want to check the status of your job use the Ganga command:

  $ In [3]: gridJob

If you want book keeping information simply look for the id string which takes the form:

  $ https://svr023.gla.scotgrid.ac.uk:9000/T2tfzxgeX6JXxq86KgSA

Then exit ganga and run

  $ edg-job-status https://svr023.gla.scotgrid.ac.uk:9000/zFIpb-Y1BgiU_EEKM9Hp9g


*************************************************************
BOOKKEEPING INFORMATION:

Status info for the Job : https://svr023.gla.scotgrid.ac.uk:9000/zFIpb-Y1BgiU_EEKM9Hp9g
Current Status:     Scheduled 
Status Reason:      Job successfully submitted to Globus
Destination:        e5grid05.physik.uni-dortmund.de:2119/jobmanager-lcgpbs-dteam
reached on:         Wed Jun 13 13:40:21 2007
*************************************************************

This will return job information information, in this case explaining that your job is scheduled to run. Once the job is completed the query will return

-bash-2.05b$ edg-job-status https://svr023.gla.scotgrid.ac.uk:9000/zFIpb-Y1BgiU_EEKM9Hp9g


*************************************************************
BOOKKEEPING INFORMATION:

Status info for the Job : https://svr023.gla.scotgrid.ac.uk:9000/zFIpb-Y1BgiU_EEKM9Hp9g
Current Status:     Cleared 
Status Reason:      user retrieved output sandbox
Destination:        e5grid05.physik.uni-dortmund.de:2119/jobmanager-lcgpbs-dteam
reached on:         Wed Jun 13 14:37:24 2007
*************************************************************

And you can collect your output.

You can also check on the output in Ganga once the job is completed:

In [7]:!cat $gridJob.outputdir/stdout

The exclamation mark before the cat command above is the way to tell IPYTHON that you issue a shell command. The dollar sign indicates that you return to a python command. By default, ganga will store jobs' outputs in ~/gangadir/workspace/Local/JOB_ID/output, where JOB_ID is a sequential job number.

This mechanism is called an input sandbox and is designed to handle small files (up to few MB). More information about handling files can be found in the ganga documentation.

Running Multiple jobs with Ganga

If you wish to run multiple jobs on the Grid with Ganga you can use a simple loop structure. The structure for submitting 100 jobs is given here:

In [24]:a=list()

In [25]:for i in range(100):
   ....:     a.append(Executable(exe='/bin/echo', args=[str(i)]))
   ....:     

In [26]:s=ExeSplitter(apps=a)

In [27]:j=Job(splitter=s,backend=LCG())

In [28]:j.submit()