Search results

Create the page "Batch System" on this wiki!

Tier1 Operations Report 2018-01-17

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,631 words) - 13:39, 17 January 2018
Past Ticket Bulletins 2018

...rrors at Edinburgh. Andy is reckoning this is a consistency problem as the system tries to delete files that aren't there anymore, and has asked if it's lots ...nk the problem might on the ECHO side now, as I suspect the changes to the batch environment have been rolled out (which caused the recent change in error m

141 KB (22,376 words) - 17:35, 21 January 2019
Tier1 Operations Report 2018-01-31

... Jan). Staff were called late Wednesday evening but were unable to get the system up then. Overnight we were unable to mount tapes - effectively blocking tap 

19 KB (1,995 words) - 13:13, 7 February 2018
Tier1 Operations Report 2018-02-07

...have noticed disproportionately large numbers of Alice jobs running on the batch farm – blocking out other VOs. Steps have been taken to correct this. * The Hyper-K VO has been enabled on the batch farm.

17 KB (1,742 words) - 14:32, 7 February 2018
Tier1 Operations Report 2018-02-13

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,609 words) - 14:04, 14 February 2018
Tier1 Operations Report 2018-03-14

...he start of this week a problem was found in which a race condition in our system configuration utility (Quattor) was causing the IPv6 configuration not to b 

17 KB (1,688 words) - 09:57, 21 March 2018
Tier1 Operations Report 2018-03-06

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,553 words) - 11:09, 6 March 2018
Tier1 Operations Report 2018-03-07

...rvice. One issue was identified – the load balancers in front of the FTS system were not dual stacked. This was fixed quickly but the problem persists. Som 

17 KB (1,682 words) - 11:38, 12 March 2018
Tier1 Operations Report 2018-03-27

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,558 words) - 09:30, 28 March 2018
Enable Queues on ARC HTCondor

One of the major differences between a standard CREAM+Torque/maui system and ARC-CE+HTCondor is the lack of queues to partition the nodes. ... particular the BDII portion is not really coded to handle it if the batch system underneath doesn't have a queue concept. To make queues work I had to patch

8 KB (1,326 words) - 07:19, 28 March 2018
Tier1 Operations Report 2018-03-21

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,716 words) - 10:14, 22 March 2018
Tier1 Operations Report 2018-03-28

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,558 words) - 09:36, 28 March 2018
Tier1 Operations Report 2018-04-04

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,577 words) - 13:16, 4 April 2018
Tier1 Operations Report 2018-04-16

...ere some problems after this. These were traced to a race condition in our system configuration utility (Quattor) that was causing the IPv6 configuration not 

18 KB (1,859 words) - 13:38, 16 April 2018
Tier1 Operations Report 2018-04-23

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

18 KB (1,796 words) - 14:47, 23 April 2018
Tier1 Operations Report 2018-04-30

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

19 KB (1,908 words) - 07:39, 1 May 2018
Tier1 Operations Report 2018-05-07

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,648 words) - 09:49, 9 May 2018
Tier1 Operations Report 2018-05-21

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,545 words) - 08:52, 30 May 2018
Tier1 Operations Report 2018-05-28

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,604 words) - 12:45, 30 May 2018
Tier1 Operations Report 2018-06-04

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,578 words) - 10:35, 6 June 2018
Tier1 Operations Report 2018-07-23

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,572 words) - 10:33, 25 July 2018
Tier1 Operations Report 2018-07-02

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,695 words) - 12:24, 11 July 2018
Tier1 Operations Report 2018-06-11

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,613 words) - 14:52, 13 June 2018
Tier1 Operations Report 2018-06-25

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,527 words) - 11:53, 27 June 2018
Tier1 Operations Report 2018-07-16

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

15 KB (1,462 words) - 10:32, 17 July 2018
Tier1 Operations Report 2018-08-06

.... The Production team experienced the intermittent working of its SMS system (the cause was finally found to be a faulty SIM). This meant that there wa ... new SIM (an associated contract), to replace the failed EE SIM in the SMS system.

16 KB (1,552 words) - 08:12, 8 August 2018
Tier1 Operations Report 2018-07-30

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,581 words) - 12:11, 1 August 2018
Tier1 Operations Report 2018-08-13

... errors. Problem identified requiring some machines to be re-installed. Batch farm at half capacity while this is being done. ... new SIM (an associated contract), to replace the failed EE SIM in the SMS system.

19 KB (1,944 words) - 13:25, 22 August 2018
Tier1 Operations Report 2018-08-20

...a precaution for the weekend, we limited the ATLAS (and CMS), quota on our batch farm to 50% of its nominal amount. Assuming we encounter no problems we int 

16 KB (1,669 words) - 13:24, 22 August 2018
Tier1 Operations Report 2018-08-27

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,734 words) - 08:33, 31 August 2018
Tier1 Operations Report 2018-09-10

... currently working on an issue with Atlas not using/being given their full batch farm allocation. 

17 KB (1,670 words) - 10:05, 11 September 2018
Tier1 Operations Report 2018-09-17

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,635 words) - 09:54, 18 September 2018
Tier1 Operations Report 2018-09-24

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,732 words) - 10:02, 26 September 2018
Tier1 Operations Report 2018-11-20

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,739 words) - 12:43, 21 November 2018
Tier1 Operations Report 2018-10-02

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,646 words) - 14:01, 3 October 2018
Tier1 Operations Report 2018-10-08

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

18 KB (1,780 words) - 10:56, 10 October 2018
Tier1 Operations Report 2018-10-15

* Occasional repeat of a CMS error – “500-globus_xio: System error in bind: Address already in use.” - This is not a major issue but 

16 KB (1,615 words) - 09:55, 16 October 2018
Tier1 Operations Report 2018-11-13

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,604 words) - 08:57, 14 November 2018
Tier1 Operations Report 2018-10-29

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,684 words) - 12:39, 31 October 2018
Tier1 Operations Report 2018-10-23

* The batch-farm is currently (23/10/18), running at reduced capacity (~20%) to facilit 

18 KB (1,757 words) - 07:53, 25 October 2018
Tier1 Operations Report 2018-11-06

* Some (1-5%) of CMS gridFTP SAM test jobs are failing against Echo due to "System error in bind: Address already in us”. This is when GridFTP can’t find 

19 KB (2,000 words) - 10:42, 6 November 2018
Tier1 Operations Report 2018-11-27

...pes and a 200 000 file backlog built up before this was noticed. The tape system priorities writing to tape above recalls (to ensure data is safe), and ther 

18 KB (1,854 words) - 13:22, 28 November 2018
Tier1 Operations Report 2018-12-17

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,587 words) - 11:20, 19 December 2018
Tier1 Operations Report 2018-12-03

* The batch farm was drained and rebooted, to apply the security patch for CVE-2018-189 

16 KB (1,565 words) - 12:59, 5 December 2018
Tier1 Operations Report 2018-12-10

* The physical machine hosting MySQL databases for the Tier-1 (RT ticket system + LFC) died on Thursday. The service was restored from backup on Friday on 

16 KB (1,608 words) - 14:28, 10 December 2018
Tier1 Operations Report 2019-01-02

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

18 KB (1,744 words) - 14:32, 2 January 2019
Tier1 Operations Report 2019-01-07

* 28th December, File System problem with one of the Squids, was fixed on 2nd Jan. 

16 KB (1,504 words) - 10:15, 9 January 2019
Tier1 Operations Report 2019-01-14

...r of LHCb MCFastSimulation jobs that are probably creating the load on the system. 

16 KB (1,491 words) - 09:54, 16 January 2019
Tracing Jobs

In a CREAM/SGE setting jobs will have both a batch system ID (a seven digit number) or the CREAM ID (a prefix, usually cream_, and an

6 KB (1,046 words) - 09:33, 5 March 2019
Tier1 Operations Report 2019-01-21

...rk issue over the weekend that impacted cvmfs. As such as we took a hit on batch farm CPU efficiencies. The issue was resolved but as the efficiencies are 

16 KB (1,619 words) - 14:40, 28 January 2019
Tier1 Operations Report 2019-01-28

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,668 words) - 12:56, 30 January 2019
Tier1 Operations Report 2019-02-18

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,629 words) - 14:06, 20 February 2019
Tier1 Operations Report 2019-02-11

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,607 words) - 13:11, 13 February 2019
Tier1 Operations Report 2019-02-05

... This was a 14 generation machine (dual purpose for Ceph). The operating system was put on the SSD (to leave other disks for capacity), which is attached t 

18 KB (1,871 words) - 11:53, 6 February 2019
Example Build of an HTCondor-CE Cluster

...r putting in front of a HTCondor cluster. But it is not tied to that batch system, and can support others. ...E installation. For simplicity, I have chosen to use HTCondor as the batch system behind the CE.

16 KB (2,635 words) - 15:21, 7 November 2019
Tier1 Operations Report 2019-06-24

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

15 KB (1,498 words) - 09:18, 26 June 2019
Tier1 Operations Report 2019-03-04

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,506 words) - 13:24, 4 March 2019
Tier1 Operations Report 2019-03-18

...sfully running batch work or accessing our storage. We do see the incoming batch jobs and we are investigating why these seem not to have run. As yet we hav 

15 KB (1,482 words) - 13:05, 20 March 2019
Tier1 Operations Report 2019-03-11

...sfully running batch work or accessing our storage. We do see the incoming batch jobs and we are investigating why these seem not to have run. As yet we hav 

15 KB (1,501 words) - 10:21, 19 March 2019
Tier1 Operations Report 2019-03-26

* Worker nodes on the batch farm had been kernel patched and rebooted. The Viglen 2011 tranche of machi 

17 KB (1,768 words) - 09:51, 27 March 2019
Tier1 Operations Report 2019-04-01

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,609 words) - 08:38, 9 April 2019
Tier1 Operations Report 2019-04-29

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

15 KB (1,517 words) - 08:55, 30 April 2019
Tier1 Operations Report 2019-04-08

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

17 KB (1,725 words) - 12:45, 10 April 2019
Tier1 Operations Report 2019-04-15

...10th April, gdss811 (LHCb) had a failure of the disk running the operating system. This generation of hardware has OS disks that are very inconveniently loc 

18 KB (1,894 words) - 12:14, 17 April 2019
Tier1 Operations Report 2019-04-22

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,649 words) - 07:37, 24 April 2019
Tier1 Operations Report 2019-05-06

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

15 KB (1,440 words) - 08:02, 14 May 2019
Tier1 Operations Report 2019-05-20

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

16 KB (1,597 words) - 09:03, 21 May 2019
Tier1 Operations Report 2019-05-14

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

15 KB (1,495 words) - 12:04, 15 May 2019
Tier1 Operations Report 2019-06-03

 ...; padding-top: 0.1em; padding-bottom: 0.1em;" | Limits on concurrent batch system jobs.

15 KB (1,464 words) - 08:37, 4 June 2019
Tier1 Operations Report 2019-06-10

...on Thursday. All SAM tests failed until this was fixed the next morning. Batch farm also did not start any new jobs during this time. We used this accide 

15 KB (1,512 words) - 08:52, 11 June 2019
GridPP approved VOs new

|MAGIC is a system of two imaging atmospheric Cherenkov telescopes (or IACTs). MAGIC-I started * high priority in the batch system for the atlassgm user;

77 KB (12,989 words) - 12:01, 19 August 2022

Search results

Navigation menu

Personal tools

Namespaces

Variants

Views

Actions

Search

Main GridPP website

Navigation

Tools