After the bgarb fix I was able to load up 11 cpu's
with gsim_bat jobs at 3 pm yesterday, using Will's
version 6 ffread/celeg proposal. As of this morning
3 of those jobs have gone into infinite loops. See the
log files below:
-rw-rw-r-- 1 lcs1h lcs1h 89804 Apr 20 10:02 apr19_7.log
-rw-rw-r-- 1 lcs1h lcs1h 85836 Apr 20 10:00 apr19_1.log
-rw-rw-r-- 1 lcs1h lcs1h 99488 Apr 20 10:00 apr19_8.log
-rw-rw-r-- 1 lcs1h lcs1h 93914 Apr 20 10:00 apr19_6.log
-rw-rw-r-- 1 lcs1h lcs1h 93926 Apr 20 10:00 apr19_3.log
-rw-rw-r-- 1 lcs1h lcs1h 105676 Apr 20 09:56 apr19_5.log
-rw-rw-r-- 1 lcs1h lcs1h 96796 Apr 20 09:55 apr19_10.log
-rw-rw-r-- 1 lcs1h lcs1h 96016 Apr 20 09:54 apr19_11.log
-rw-rw-r-- 1 lcs1h lcs1h 99004 Apr 20 08:29 apr19_9.log
-rw-rw-r-- 1 lcs1h lcs1h 92284 Apr 20 06:23 apr19_4.log
-rw-rw-r-- 1 lcs1h lcs1h 77286 Apr 20 01:31 apr19_2.log
Jobs on nodes 2 4 and 9 stopped writing to disk. The log files do
not give me any hints.
I'm getting about 1.3 s/event/node so I should be able to achieve
700K events/day with 100% availability. Unfortunately I'm sharing
a pvm job so I'm down to 350K/day at the moment, not counting the
dead jobs.
Cole
cole@apollo.phys.virginia.edu