LSST archived actions
From GridPP Wiki
Action ID | Action description | Owner | Target date | Status | Last update | Notes |
---|---|---|---|---|---|---|
VOI-LSST-001 | Ganga direct job submission using Northgrid | AF, JZ | 2015-01-31 | Closed | 2015-12-11 | Do direct job submission testing using northgrid infrastructure and ganga |
VOI-LSST-002 | Get European users on the LSST VOMS server at FNAL | AF, JZ | 2015-02-28 | Closed | 2015-12-11 | |
VOI-LSST-003 | Enable LSST at sites | AF, AW, EM, SJ | 2015-06-30 | Closed | 2015-12-11 | Get the correct configuration. Sites get the info from Operations portal which has some obsolete information about VOMS. We should ask to fix it. |
VOI-LSST-004 | Find which LSST CVMFS stratum0 is usable by us | AF | 2015-08-10 | Closed | 2015-12-11 | 3 instances, in France, OSG and FNAL. Chosen FNAL |
VOI-LSST-005 | Get the repository at FNAL and replicated at RAL | AF, CC | 2015-08-25 | Closed | 2015-12-11 | Repository automatically mounted at EGI sites as part of OSG EGI agreement, but was not replicated on any EGI stratum1. https://ggus.eu/index.php?mode=ticket_info&ticket_id=115335
|
VOI-LSST-006 | Run jobs using LSST CVMFS using direct job submission and ganga | JZ | 2015-09-30 | Closed | 2015-12-11 | Joe uploaded the software and used it to run jobs with direct job submission using lsst VO |
VOI-LSST-007 | Enable LSST on Dirac | AF, DB, AW, EM, SJ | 2015-09-30 | Closed | 2015-12-11 | Got the Dirac pilot DN assigned to the pilot role on VOMS, enabled the VO on Dirac, enabled pilot at sites, tested submission and fixed misconfigured sites. JIRA 1) https://its.cern.ch/jira/browse/GRIDPP-22 2) https://ggus.eu/index.php?mode=ticket_info&ticket_id=117585 3) https://ggus.eu/index.php?mode=ticket_info&ticket_id=117586 |
VOI-LSST-008 | Test Ganga Dirac setup | AF | Closed | 2015-12-11 | 1) https://its.cern.ch/jira/browse/GRIDPP-22 2) https://github.com/ganga-devs/ganga/issues/45 3) https://its.cern.ch/jira/browse/GRIDPP-29 since gfal_util doesn't work instead of debugging it I've started to look into the dirac file catalogue clients, which have better compatiblity chances on top of having the file catalogue. See VO-LSST-014 | |
VOI-LSST-009 | voms-proxy-init EMI-3 not working | AF, DB | Closed | 2015-04-05 | FNAL upgraded the VOMS to a EMI-3 clients compatible version. Ticket https://ggus.eu/index.php?mode=ticket_info&ticket_id=117587 closed. Ticket for VOMS developers: https://ggus.eu/index.php?mode=ticket_info&ticket_id=114044 closed. | |
VOI-LSST-010 | Update the Operations portal with correct VOMS info | AF | Closed | 2015-01-05 | LSST US managers added voms1 and voms2 to the ops portal. http://operations-portal.egi.eu/vo/view/voname/lsst | |
VOI-LSST-011 | Long lived proxies | AF | Closed | 2016-04-05 | Ticket with FNAL RITM0302478 was closed after we moved to dirac and we could run jobs longer than 24h with the renwal mechanism. | |
VOI-LSST-012 | Check with NERSC how to use gridftp, authentication and authorisation mechanisms | JZ, ME, AF | Closed | 2016-01-11 | NERSC account required to do the transfers. Marcus has now an account. | |
VOI-LSST-013 | Test gridtp transfers with NERSC | JZ, ME | Closed | 2016-01-25 | Data copied from NERSC with globus-url-copy | |
VOI-LSST-014 | Investigate Dirac file catalogue usage | AF, JZ | Closed | 2016-02-22 | https://its.cern.ch/jira/browse/GRIDPP-30 More info in https://groups.google.com/forum/#!topic/diracgrid-forum/sclcLrQBPFY. Files transferred to Manchester with correct directory naming scheme is working. See VOI-LSST-015. | |
VOI-LSST-015 | Spread the LSST data on different sites | AF | Closed | 2016-11-22 | https://its.cern.ch/jira/browse/GRIDPP-31 This was completed in May | |
VOI-LSST-016 | Move to use gridpp VO because GridPP management requested to run an analysis with any "means possible" | AF, JZ, RF, DB, KM | Closed | 2016-04-05 | We went back to LSST and dirac | |
VOI-LSST-017 | Edinburgh LSST data access | ME | Closed | 2016-02-08 | The local time out problem with the data access was solved and data is available now for jobs and transfers without timing out | |
VOI-LSST-018 | Adapt Joe's ganga and bash scripts to submit to dirac and use the dirac file catalog clients instead of gfal_utils. | AF | Closed | 2016-04-05 | AF adapted Joe's scripts and have been running most of the first part of data. The new data will be processed with a completely revised ganga script after the involvement of ganga developers. | |
VOI-LSST-019 | Run a larger sample of jobs (effectively 2500) | AF, JZ | Closed | 2016-11-22 | https://its.cern.ch/jira/browse/GRIDPP-33 run another batch of 5000 jobs but mostly had a timeout problem when downloading the input. It is not clear if the problem is from the DFC or the storages. We hope the new ganga client can get better error codes and solve this more easily. Ticket for this https://github.com/ganga-devs/ganga/issues/343. | |
VOI-LSST-020 | Implement JZ workflow in plain dirac cli | ME | Closed | 2016-11-29 | dirac cli is implemented in scripts and usable for job submission and getting OutputSandbox using a given input file list, a test running over 1000 input files was successful, needs to be tested for overall analysis usage by JZ - was tested to work some month ago, available for future use if needed | |
VOI-LSST-021 | Copy new data from NERSC to one of the storage elements and register them in the DFC | JZ, AF | Closed | 2016-04-30 | Data have been copied to Liverpoool and registered. | |
VOI-LSST-022 | Test gfal and dirac utils on NERSC | ME, JZ | Closed | 2016-04-29 | Dirac UI can be installed in the user's $HOME on the data transfer machines at NERSC (RH6 OS installed) and then directly be used to transfer the data and register the files in the catalogue in a single step. | |
VOI-LSST-023 | Enable LSST at RAL, QMUL, Glasgow | AF, DT, AL, GR | Closed | 2016-11-22 | Tickets opened for sites: Sites enabled LSST and were tested with LSST jobs. https://ggus.eu/?mode=ticket_info&ticket_id=120352 https://ggus.eu/?mode=ticket_info&ticket_id=120351 https://ggus.eu/?mode=ticket_info&ticket_id=120350 | |
VOI-LSST-024 | Debug Lancaster failures | AF, MD | Closed. | 2016-11-22 | More recent tests were successful. | |
VOI-LSST-025 | USLSST Panda job submission | AF, PL, PS | Closed. | 2017-09-26 | Work ongoing with BNL to run jobs submitted via panda. This is now working at Lancaster, Manchester and RAL. |