LSST archived actions

From GridPP Wiki
Jump to: navigation, search
Action ID Action description Owner Target date Status Last update Notes
VOI-LSST-001 Ganga direct job submission using Northgrid AF, JZ 2015-01-31 Closed 2015-12-11 Do direct job submission testing using northgrid infrastructure and ganga
VOI-LSST-002 Get European users on the LSST VOMS server at FNAL AF, JZ 2015-02-28 Closed 2015-12-11
VOI-LSST-003 Enable LSST at sites AF, AW, EM, SJ 2015-06-30 Closed 2015-12-11 Get the correct configuration. Sites get the info from Operations portal which has some obsolete information about VOMS. We should ask to fix it.
VOI-LSST-004 Find which LSST CVMFS stratum0 is usable by us AF 2015-08-10 Closed 2015-12-11 3 instances, in France, OSG and FNAL. Chosen FNAL
VOI-LSST-005 Get the repository at FNAL and replicated at RAL AF, CC 2015-08-25 Closed 2015-12-11 Repository automatically mounted at EGI sites as part of OSG EGI agreement, but was not replicated on any EGI stratum1. https://ggus.eu/index.php?mode=ticket_info&ticket_id=115335


VOI-LSST-006 Run jobs using LSST CVMFS using direct job submission and ganga JZ 2015-09-30 Closed 2015-12-11 Joe uploaded the software and used it to run jobs with direct job submission using lsst VO
VOI-LSST-007 Enable LSST on Dirac AF, DB, AW, EM, SJ 2015-09-30 Closed 2015-12-11 Got the Dirac pilot DN assigned to the pilot role on VOMS, enabled the VO on Dirac, enabled pilot at sites, tested submission and fixed misconfigured sites. JIRA
1) https://its.cern.ch/jira/browse/GRIDPP-22
2) https://ggus.eu/index.php?mode=ticket_info&ticket_id=117585
3) https://ggus.eu/index.php?mode=ticket_info&ticket_id=117586
VOI-LSST-008 Test Ganga Dirac setup AF Closed 2015-12-11 1) https://its.cern.ch/jira/browse/GRIDPP-22
2) https://github.com/ganga-devs/ganga/issues/45
3) https://its.cern.ch/jira/browse/GRIDPP-29

since gfal_util doesn't work instead of debugging it I've started to look into the dirac file catalogue clients, which have better compatiblity chances on top of having the file catalogue. See VO-LSST-014

VOI-LSST-009 voms-proxy-init EMI-3 not working AF, DB Closed 2015-04-05 FNAL upgraded the VOMS to a EMI-3 clients compatible version. Ticket https://ggus.eu/index.php?mode=ticket_info&ticket_id=117587 closed. Ticket for VOMS developers: https://ggus.eu/index.php?mode=ticket_info&ticket_id=114044 closed.
VOI-LSST-010 Update the Operations portal with correct VOMS info AF Closed 2015-01-05 LSST US managers added voms1 and voms2 to the ops portal. http://operations-portal.egi.eu/vo/view/voname/lsst
VOI-LSST-011 Long lived proxies AF Closed 2016-04-05 Ticket with FNAL RITM0302478 was closed after we moved to dirac and we could run jobs longer than 24h with the renwal mechanism.
VOI-LSST-012 Check with NERSC how to use gridftp, authentication and authorisation mechanisms JZ, ME, AF Closed 2016-01-11 NERSC account required to do the transfers. Marcus has now an account.
VOI-LSST-013 Test gridtp transfers with NERSC JZ, ME Closed 2016-01-25 Data copied from NERSC with globus-url-copy
VOI-LSST-014 Investigate Dirac file catalogue usage AF, JZ Closed 2016-02-22 https://its.cern.ch/jira/browse/GRIDPP-30
More info in
https://groups.google.com/forum/#!topic/diracgrid-forum/sclcLrQBPFY.
Files transferred to Manchester with correct directory naming scheme is working. See VOI-LSST-015.
VOI-LSST-015 Spread the LSST data on different sites AF Closed 2016-11-22 https://its.cern.ch/jira/browse/GRIDPP-31
This was completed in May
VOI-LSST-016 Move to use gridpp VO because GridPP management requested to run an analysis with any "means possible" AF, JZ, RF, DB, KM Closed 2016-04-05 We went back to LSST and dirac
VOI-LSST-017 Edinburgh LSST data access ME Closed 2016-02-08 The local time out problem with the data access was solved and data is available now for jobs and transfers without timing out
VOI-LSST-018 Adapt Joe's ganga and bash scripts to submit to dirac and use the dirac file catalog clients instead of gfal_utils. AF Closed 2016-04-05 AF adapted Joe's scripts and have been running most of the first part of data. The new data will be processed with a completely revised ganga script after the involvement of ganga developers.
VOI-LSST-019 Run a larger sample of jobs (effectively 2500) AF, JZ Closed 2016-11-22 https://its.cern.ch/jira/browse/GRIDPP-33 run another batch of 5000 jobs but mostly had a timeout problem when downloading the input. It is not clear if the problem is from the DFC or the storages. We hope the new ganga client can get better error codes and solve this more easily. Ticket for this https://github.com/ganga-devs/ganga/issues/343.
VOI-LSST-020 Implement JZ workflow in plain dirac cli ME Closed 2016-11-29 dirac cli is implemented in scripts and usable for job submission and getting OutputSandbox using a given input file list, a test running over 1000 input files was successful, needs to be tested for overall analysis usage by JZ - was tested to work some month ago, available for future use if needed
VOI-LSST-021 Copy new data from NERSC to one of the storage elements and register them in the DFC JZ, AF Closed 2016-04-30 Data have been copied to Liverpoool and registered.
VOI-LSST-022 Test gfal and dirac utils on NERSC ME, JZ Closed 2016-04-29 Dirac UI can be installed in the user's $HOME on the data transfer machines at NERSC (RH6 OS installed) and then directly be used to transfer the data and register the files in the catalogue in a single step.
VOI-LSST-023 Enable LSST at RAL, QMUL, Glasgow AF, DT, AL, GR Closed 2016-11-22 Tickets opened for sites: Sites enabled LSST and were tested with LSST jobs.
https://ggus.eu/?mode=ticket_info&ticket_id=120352
https://ggus.eu/?mode=ticket_info&ticket_id=120351
https://ggus.eu/?mode=ticket_info&ticket_id=120350
VOI-LSST-024 Debug Lancaster failures AF, MD Closed. 2016-11-22 More recent tests were successful.
VOI-LSST-025 USLSST Panda job submission AF, PL, PS Closed. 2017-09-26 Work ongoing with BNL to run jobs submitted via panda. This is now working at Lancaster, Manchester and RAL.