APEL Java Out Of Memory Error

This was the proceedure I used at RALPP to fill in a gap in the APEL accounting log which was causing APEL to fail with a JAVA Out of Memeory error while trying to upload new statistics. It is based on a recipe sent to me by Rob Byrom (his instructions are in blue with my commetary in black).

1) Edit the publisher-config.xml file (this is the config file located on your apel 'publisher' ie the host where the accounting records are published from).

On my yaim configured machine this was actually /opt/glite/etc/glite-apel-publisher/publisher-config-yaim.xml

2) Set Republish = nothing

3) Re-run the apel application (this should only publish the new records for the day).

4) Take a copy of your mysql database:

Mysqldump accounting LcgRecords --no-create-info > dump.txt

This assumes your database is called accounting. The output is stuffed into dump.txt (but double check it does contain plausible data!).

5) Go into you accounting database and :

SELECT * FROM RepublishInfo;

This should give you the last date when your site's accounting data was in synch with the GOC server.

This didn't do that for me it only gave me today's date. I used the chart at http://goc.grid-support.ac.uk/gridsite/accounting/tree/gridpp_view.php?BASE=NormSumCPU&startYear=2005&startMonth=10&endYear=2006&endMonth=1&ExecutingSite=RALPP-LCG&Submit=Refresh to judge the missing records

6) Using the date from 5) delete the accounting records that occur before this timestamp from the LccRecords table (I would add-in an extra margin of error of a couple of weeks before, just to be safe).

delete from LcgRecords where EventDate<"2005-12-14";

7) Edit your publisher-config.xml file and set the Republish = all.

8) Re-run the apel publisher, it should then attempt to publish all the contents under the LcgRecords (but with a much smaller subset of data).

9) Edit your publisher-config.xml file and set the Republish = missing.

10) Re-run the apel publisher, this time it should tell you all records were successfully updated (the warnings are fairly obvious).

11) Dump the contents of the dump.txt back into the database:

Mysql accounting < dump.txt

If it complains about duplicate rows, you may need to delete the LcgRecords table contents, then retry.

I got these errors and so needed to:

delete from LcgRecords;

You might need several goes at this if there are a large ammount of records to republish.

All the mysql was done as the mysql root user without any problems.

This woked for me - your milage my vary.