Hi Nathan, These are the last jobs of the 10% pass. Usually yes I am keeping the queue, I am not waiting all the jobs to be finished before submitting the next chunk. Another thing that is starting to annoy more and more, is that ifarms are becoming practically unusable, most of the time they are overloaded, and a simple "ls" command takes forever. This happens to "/work" disk too. This behavior impacts on tarring, i.e. farms put a lot of outputs to work disk, but ifarms are "Too slow" yo catch up with farm jobs. One thing I was thinking, will be good if hps would have a separate machine designed for tarring files and sending to tape, or even would be much better if that machine would have about 20T free space, that we will be separate from the work disk. I don't know how reasonable is this... Rafo ________________________________ From: Nathan Baltzell Sent: Thursday, February 7, 2019 9:47:27 PM To: Rafayel Paremuzyan Cc: HPS-SOFTWARE Subject: Re: The new cooking Hi Rafo, Based on scicomp.jlab.org<http://scicomp.jlab.org> website, I see ~800 hps jobs in SLURM currently running (great, almost 3x more than hps's previous *average* experience on jlab batch farm) but almost none in the queue or in depend state. Wondering if it may be better to keep the queue more saturated? I guess you are staging things in batches (including tarballing to tape)? Ok, I was just taking a look and curious about throughput:) -Nathan On Feb 6, 2019, at 23:52, Rafayel Paremuzyan <[log in to unmask]<mailto:[log in to unmask]>> wrote: Hi all, while the new pass BLPass4b is being cooked, you could look into pass related details in this confluence page. https://confluence.slac.stanford.edu/display/hpsg/BLPass4b<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fconfluence.slac.stanford.edu%2Fdisplay%2Fhpsg%2FBLPass4b&data=02%7C01%7Crafopar%40jlab.org%7C7c037afc788d4250a07c08d68d6fc385%7Cb4d7ee1f4fb34f0690372b5b522042ab%7C1%7C0%7C636851908496839348&sdata=5yzwPjQmkiqt8selrtCLEo9eqMdr%2BPRIU2mV72gzqzE%3D&reserved=0>, Rafo ________________________________ Use REPLY-ALL to reply to list To unsubscribe from the HPS-SOFTWARE list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flistserv.slac.stanford.edu%2Fcgi-bin%2Fwa%3FSUBED1%3DHPS-SOFTWARE%26A%3D1&data=02%7C01%7Crafopar%40jlab.org%7C7c037afc788d4250a07c08d68d6fc385%7Cb4d7ee1f4fb34f0690372b5b522042ab%7C1%7C0%7C636851908496849348&sdata=CSpqTuN1EQxGu4K1pTicRGN9qX26qr7IIYhQuPX%2F0XI%3D&reserved=0> ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the HPS-SOFTWARE list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=HPS-SOFTWARE&A=1