Difference between revisions of "GridPP VO Incubator"
(→LSST) |
(→LSST) |
||
Line 473: | Line 473: | ||
| VOI-LSST-19 | | VOI-LSST-19 | ||
| Run a larger sample of jobs (effectively 2500) | | Run a larger sample of jobs (effectively 2500) | ||
− | | AF | + | | AF, JZ |
| | | | ||
| In progress | | In progress |
Revision as of 12:30, 25 February 2016
This page is for monitoring the progress of new(ish) GridPP VOs.
- PoC - Point of Contact
Contents
All new VOs
These tasks will need to be completed for all
Action ID | Action description | Owner | Target date | Status | Date closed | Notes |
---|---|---|---|---|---|---|
VOI-GEN-001 | Deploy test software to RVO CernVM-FS repositories. | Duncan, Daniela, Gareth, Alessandra, Ewan | 2015-05-26 | Open | N/A | New users in the Regional VOs will need to run test jobs using software in the RVO CernVM-FS repositories. This test software will need to be uploaded by the RVO admins. Instructions for doing this can be found here. |
VOI-GEN-002 | Write up the VO registration procedure | Tom | 2015-05-31 | In progress | N/A | Guide started here - comments and feedback appreciated. |
DEAP3600
PoC: Jeremy Coles (JC)
- Update requested February 2016.
vo.DiRAC.ac.uk
PoC: Jens Jensen, Brian Davies
Action ID | Action description | Owner | Target date | Status | Date closed | Notes |
---|---|---|---|---|---|---|
VOI-DIRAC-001 | Set up VO in GridPP | JJ | 2015-04-30 | Closed | ||
VOI-DIRAC-002 | Register DiRAC with EGI | JJ | 2015-05-31 | Closed | [1] | |
VOI-DIRAC-003 | Write up DiRAC site setup document | LH | 2015-08-31 | Closed | Version 1.3 circulated to DIRAC-USERS | |
VOI-DIRAC-004 | Re-evaluate data packaging method | JJ | 2015-11-30 | In Progress | Need a new method. Sam contributing. | |
VOI-DIRAC-005 | Restart transfers with new method (VOI-DIRAC-004) | LH | 2015-12-10 | In Progress | Successful large data transfer over new year 2016, http://gridpp-storage.blogspot.co.uk/2016/01/update-on-vodiracacuk-data-mopvement.html slight error in script (leading to some failures) being updated | |
VOI-DIRAC-006 | Get Leicester ready for transfer | JJ | 2015-11-30 | In Progress | Leicester now have certificates again. Need to await solution to VO-DIRAC-004 | |
VOI-DIRAC-007 | Request or requirement for ACL between sites. | BD | 2015-11-30 | In Progress | Awaiting to here if it is a request or a requirement to keep data access seperate bewteen DIRAC sites. I fneeded new voms roles and castor setup may need to be enabled.
|
EUCLID
PoC: Andrew Lahiff (AL)
Action ID | Action description | Owner | Target date | Status | Date closed | Notes |
---|---|---|---|---|---|---|
VOI-EUC-001 | Enable /cvmfs/euclid.in2p3.fr on RAL worker nodes | AL | 2016-02-24 | Open | ||
VOI-EUC-002 | Setup /cvmfs/euclid-uk.egi.eu repository | CC | 2016-02-26 | Open | ||
VOI-EUC-003 | Setup accounts on RAL UIs | AL | 2016-02-26 | Open |
GalDyn
- PoC: Tom Whyntie (TW)
- UCLan: Adam Clarke (AC)
Action ID | Action description | Owner | Target date | Status | Date closed | Notes |
---|---|---|---|---|---|---|
VOI-GAL-001 | Assist GalDyn users with CernVM creation and testing. | TW | 2015-02-18 | Closed | 2015-02-18 | GalDyn users have successfully instantiated CernVMs for accessing the grid. |
VOI-GAL-002 | Assist GalDyn users with running test jobs on the Imperial DIRAC instance. | TW | 2015-02-23 | Closed | 2015-02-23 | GalDyn users have successfully run test jobs on the Imperial DIRAC instance via a GridPP CernVM. |
VOI-GAL-003 | Assist GalDyn users with compiling user software on the CernVM. | TW | 2015-03-08 | Closed | 1016-02-15 | The code compiles and runs, but needs to be put in a grid/CernVM-FS-friendly format. |
VOI-GAL-004 | Create the GalDyn CernVM-FS repository | TW, CC | 2015-05-05 | Closed | 2015-05-05 | New CernVM-FS repository galdyn.egi.eu has been created on the RAL Stratum-1 for the GalDyn VO. |
VOI-GAL-005 | (Re)new grid certificate for AC | AC | 2015-02-22 | In progress | AC's grid certificate expired while writing up his PhD thesis - trying to renew if possible, or apply for a new one. UK CA Support Ticket raised. |
LIGO
PoC: Catalin Condurache (CC)
LIGO: Paul Hopkins (PH)
Other people: Andrew Lahiff (AL)
Action ID | Action description | Owner | Target date | Status | Date closed | Notes |
---|---|---|---|---|---|---|
VOI-LIGO-001 | Create the LIGO CernVM-FS repository | CC | 2014-12-01 | Closed | 2015-02-15 | New CernVM-FS repository ligo.egi.eu has been created on the RAL Stratum-1 for the LIGO VO. |
VOI-LIGO-002 | Assist LIGO users with using Condor + nordugrid to access ARC-CE@RAL. | AL, CC | 2015-12 | Closed | 2016-02-12 | Test jobs submitted from LIGO Condor instance to ARC-CE service at RAL were successful. |
VOI-LIGO-003 | Plan to run proper analyses jobs using scientists involvement | PH, CC | 2016-02-12 | Open |
LOFAR
PoC: George Ryall (GR)
LSST
PoC: Alessandra Forti (AF)
Other people: Joe Zuntz (JZ), Andy Washbrook (AW), Ewan McMahon (EM), Steve Jones (SJ), Catalin Condurache (CC), Daniela Bauer (DB), Marcus Ebert (ME), Kashif Mohammad (KM)
Action ID | Action description | Owner | Target date | Status | Last update | Notes |
---|---|---|---|---|---|---|
VOI-LSST-001 | Ganga direct job submission using Northgrid | AF, JZ | 2015-01-31 | Closed | 2015-12-11 | Do direct job submission testing using northgrid infrastructure and ganga |
VOI-LSST-002 | Get European users on the LSST VOMS server at FNAL | AF, JZ | 2015-02-28 | Closed | 2015-12-11 | |
VOI-LSST-003 | Enable LSST at sites | AF, AW, EM, SJ | 2015-06-30 | Closed | 2015-12-11 | Get the correct configuration. Sites get the info from Operations portal which has some obsolete information about VOMS. We should ask to fix it. |
VOI-LSST-004 | Find which LSST CVMFS stratum0 is usable by us | AF | 2015-08-10 | Closed | 2015-12-11 | 3 instances, in France, OSG and FNAL. Chosen FNAL |
VOI-LSST-005 | Get the repository at FNAL and replicated at RAL | AF, CC | 2015-08-25 | Closed | 2015-12-11 | Repository automatically mounted at EGI sites as part of OSG EGI agreement, but was not replicated on any EGI stratum1. https://ggus.eu/index.php?mode=ticket_info&ticket_id=115335
|
VOI-LSST-006 | Run jobs using LSST CVMFS using direct job submission and ganga | JZ | 2015-09-30 | Closed | 2015-12-11 | Joe uploaded the software and used it to run jobs with direct job submission using lsst VO |
VOI-LSST-007 | Enable LSST on Dirac | AF, DB, AW, EM, SJ | 2015-09-30 | Closed | 2015-12-11 | Got the Dirac pilot DN assigned to the pilot role on VOMS, enabled the VO on Dirac, enabled pilot at sites, tested submission and fixed misconfigured sites. JIRA 1) https://its.cern.ch/jira/browse/GRIDPP-22 2) https://ggus.eu/index.php?mode=ticket_info&ticket_id=117585 3) https://ggus.eu/index.php?mode=ticket_info&ticket_id=117586 |
VOI-LSST-008 | Test Ganga Dirac setup | AF | Closed | 2015-12-11 | 1) https://its.cern.ch/jira/browse/GRIDPP-22 2) https://github.com/ganga-devs/ganga/issues/45 3) https://its.cern.ch/jira/browse/GRIDPP-29 since gfal_util doesn't work instead of debugging it I've started to look into the dirac file catalogue clients, which have better compatiblity chances on top of having the file catalogue. See VO-LSST-014 | |
VOI-LSST-009 | voms-proxy-init EMI-3 not working | AF, DB | Closed | 2015-01-11 | FNAL upgraded the VOMS to a EMI-3 clients compatible version. Ticket https://ggus.eu/index.php?mode=ticket_info&ticket_id=117587 closed. | |
VOI-LSST-010 | Update the Operations portal with correct VOMS info | AF | Closed | 2015-01-05 | LSST US managers added voms1 and voms2 to the ops portal. http://operations-portal.egi.eu/vo/view/voname/lsst | |
VOI-LSST-011 | Long lived proxies | AF | In progress | 2016-02-22 | Ticket with FNAL RITM0302478. The mechanisms of renewal and how LHC experiments do it is now understood. Using dirac may help, but this needs to be tested. See VO-LSST-18 | |
VOI-LSST-012 | Check with NERSC how to use gridftp, authentication and authorisation mechanisms | JZ, ME, AF | Closed | 2015-01-11 | NERSC account required to do the transfers. Marcus has now an account. | |
VOI-LSST-013 | Test gridtp transfers with NERSC | JZ, ME | Closed | 2015-01-25 | Data copied from NERSC with globus-url-copy | |
VOI-LSST-014 | Investigate Dirac file catalogue usage | AF, JZ | Closed | 2016-02-22 | https://its.cern.ch/jira/browse/GRIDPP-30 More info in https://groups.google.com/forum/#!topic/diracgrid-forum/sclcLrQBPFY. Files transferred to Manchester with correct directory naming scheme is working. See VOI-LSST-015. | |
VOI-LSST-015 | Spread the LSST data on different sites | AF | In progress | 2016-02-09 | https://its.cern.ch/jira/browse/GRIDPP-31 All data on a test SE instance for 20,000 foreseen jobs. Currently only 3 SEs (Ed, Man, Liv) enabled and mostly on older hardware. Replication will also allow to solve the registration problem by copying to a subdirectory <storage_path>/lsst/lsst. First leg of the distribution done. All files copied to Manchester. | |
VOI-LSST-016 | Move to use gridpp VO because GridPP management requested to run an analysis with any "means possible" | AF, JZ, RF, DB, KM | In progress | 2016-02-16 | Requires direct job submission and longer proxies. https://ggus.eu/index.php?mode=ticket_info&ticket_id=119265 https://ggus.eu/index.php?mode=ticket_info&ticket_id=119322 https://ggus.eu/index.php?mode=ticket_info&ticket_id=119323. Joe has submitted his jobs and the majority run. There were problems in Bristol and Cambridge which are under investigation. Joe was recommended to remove them from the list for now and resubmit the jobs. With this | |
VOI-LSST-17 | Edinburgh LSST data access | ME | Closed | 2016-02-08 | The local time out problem with the data access was solved and data is available now for jobs and transfers without timing out | |
VOI-LSST-18 | Adapt Joe's ganga and bash scripts to submit to dirac and use the dirac file catalog clients instead of gfal_utils. | AF | In progress | 2016-02-23 | the current scripts use gfal which is incompatible with dirac job submission. This was a stumbling block that forced us to go back to use gridpp and direct job submission. The scripts are modified and run successfully two jobs at UKI-SOUTHGRID-RALPP (ARC). I've also tested with a larger sample of 165 which is currently still running in RALPP,Liverpool and Manchester. Jobs run also in Lancs but failed there (to be investigated). 165 jobs run is almost finished. | |
VOI-LSST-19 | Run a larger sample of jobs (effectively 2500) | AF, JZ | In progress | 2016-02-25 | https://its.cern.ch/jira/browse/GRIDPP-33 |
LZ
PoC: David Colling (DC)
PRaVDA
PoC: Mark Slater/ Matt Williams (MS/MW)
- Update requested 3rd Feb., 19th Feb. 2016 by TW.
SNO+
- SNO+: Matthew Mottram (MM)
Action ID | Action description | Owner | Target date | Status | Last update | Notes |
---|---|---|---|---|---|---|
VOI-SNO+-001 | Check on progress via GridPP-Support list | PG | 2016-02-17 | On going | ||
VOI-SNO+-002 | MM to join GridPP Storage meeting | PG | 2016-02-24 | Closed | 2016-02-24 | MM joined the meeting to discuss requirements and various options. See minutes. |
SuperNEMO
PoC: Pete Gronbech
Action ID | Action description | Owner | Target date | Status | Last update | Notes |
---|---|---|---|---|---|---|
VOI-SuperNEMO-001 | Check on progress via GridPP-Support list | PG | 2016-02-17 | On going |
UKQCD
PoC: Jeremy Coles (JC)
- Update requested February 2016.