Glasgow Job Wrapper Worked Example (Deprecated)

From GridPP Wiki
Revision as of 14:24, 21 June 2007 by Graeme stewart (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Running executables with GANGA

First of all log in and initialise your proxy as normal:

  • Login to ppeui.
  • Initialise a proxy
 $ grid-proxy-init

Copying Necessary Files

  • Now copy over all the files necessary to run your job using gsisftp
 $ gsisftp -oPort=2222

This will open up sftp, in this mode you can copy over files using 'put'

  sftp> put test.exe
  Uploading test.exe to /clusterhome/home/gla019/test.exe

Remember to replace test.exe with your own files and gla019 with your username. Exit sftp using CTRL-D.

Now login to on port 222 using

 $ gsissh -p 2222

and copy the file you transfered to your shared directory

 $ cp test.exe /cluster/share/gla019/

Running Ganga

If you are new to Ganga familiarise yourself with the Glasgow_Ganga_Quickstart_Guide. When ready activate Ganga as normal:


To run on the grid use

     $ In [4]:gridJob=Job(backend=LCG(),application=Executable(exe=File('/cluster/share/gla019/test.exe'), 

This is equivalent to executing the command

   $/cluster/share/gla019/test.exe 1 2 3

and will run the job anywhere your VO is supported. However you can target specific sites if you wish, for instance to target glasgow you would use:

  $ In [5]:gridJob.backend.CE=''

if your VO is dteam. To actually submit the job use

In [6]:gridJob.submit()
Ganga.GPIDev.Lib.Job               : INFO     submitting job 3
Ganga.GPIDev.Adapters              : INFO     submitting job 3 to LCG backend
Ganga.GPIDev.Lib.Job               : INFO     job 3 status changed to "submitted"
Out[6]: 1

Checking Job Output

If you want to check the status of your job use the Ganga command:

  $ In [3]: gridJob

If you want book keeping information simply look for the id string which takes the form:


Then exit ganga and run

  $ edg-job-status


Status info for the Job :
Current Status:     Scheduled 
Status Reason:      Job successfully submitted to Globus
reached on:         Wed Jun 13 13:40:21 2007

This will return job information information, in this case explaining that your job is scheduled to run. Once the job is completed the query will return

-bash-2.05b$ edg-job-status


Status info for the Job :
Current Status:     Cleared 
Status Reason:      user retrieved output sandbox
reached on:         Wed Jun 13 14:37:24 2007

And you can collect your output.

You can also check on the output in Ganga once the job is completed:

In [7]:!cat $gridJob.outputdir/stdout

The exclamation mark before the cat command above is the way to tell IPYTHON that you issue a shell command. The dollar sign indicates that you return to a python command. By default, ganga will store jobs' outputs in ~/gangadir/workspace/Local/JOB_ID/output, where JOB_ID is a sequential job number.

This mechanism is called an input sandbox and is designed to handle small files (up to few MB). More information about handling files can be found in the ganga documentation.

Running Multiple jobs with Ganga

If you wish to run multiple jobs on the Grid with Ganga you can use a simple loop structure. The structure for submitting 100 jobs is given here:

In [24]:a=list()

In [25]:for i in range(100):
   ....:     a.append(Executable(exe='/bin/echo', args=[str(i)]))

In [26]:s=ExeSplitter(apps=a)

In [27]:j=Job(splitter=s,backend=LCG())

In [28]:j.submit()