LISTSERV mailing list manager LISTSERV 16.5

Help for VUB-RECOIL Archives


VUB-RECOIL Archives

VUB-RECOIL Archives


VUB-RECOIL@LISTSERV.SLAC.STANFORD.EDU


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

VUB-RECOIL Home

VUB-RECOIL Home

VUB-RECOIL  April 2004

VUB-RECOIL April 2004

Subject:

Production 2004: CPU and disk projection

From:

Urs Langenegger <[log in to unmask]>

Reply-To:

[log in to unmask]

Date:

14 Apr 2004 12:16:00 +0200Wed, 14 Apr 2004 12:16:00 +0200

Content-Type:

text/plain

Parts/Attachments:

Parts/Attachments

text/plain (171 lines)


Hoi,

modulo any errors  in my computations, the production  2004 for 150/fb
data and 500/fb BB MC will need

  1.3TB     disk space (+/- 150GB)
  O(60000)  CPU hours (assuming that the tagbits are functional in SP5)
  O(150000) CPU hours (worst case)

Deleting  the moments-analysis (and  other) hbook  files will  free up
O(750GB) of AWG diskspace, i.e., we'd need to find about 600GB.

Neither  scenario  is a  serious  problem  to  achieve in  two  months
(barring catastrophic SCS/batch breakdowns). See below for details.

Seems  like a good  deal to  get two  theses out  by fall  and produce
publication-ready results before  BELLE publishes their combination of
b->sg and b->ulnu as another "first" ...


Cheers,
--U.


----------------------------------------------------------------------
1. Current ISL disks
====================

We have the following AWG disks: 

 shire01>df -k  /nfs/farm/babar/AWG37/.  /nfs/farm/babar/AWG23/.  /nfs/farm/babar/AWG18/.
 Filesystem            kbytes    used   avail capacity  Mounted on
 sulky25:/AWG18       619788288 616803143 2800045   100%    /a/sulky25/AWG18
 sulky26:/AWG23       619788288 613374726 6013098   100%    /a/sulky26/AWG23
 sulky13:/AWG37       619788288 607730751 11310159    99%    /a/sulky13/AWG37

They contain a mixture of files: 
-----------------------------------------------------------------------------
Disk       User            Size          What
-----------------------------------------------------------------------------
AWG18      vub-recoil      515948923     ISL/sx-080702: gen BB MC
                                                        data 
                                                        skimmed data rfiles
           moments         - 
           miscellaneous              

AWG23      vub-recoil      107038654     ISL/sx-080702: signal MC
							cocktail MC
                                                        skimmed MC rfiles
                                                        hbook_gen_mar ???
                                         ISL/VubAna-out: DELETE ???
           moments         498613341

AWG37      vub-recoil      262869337     ISL/sx-080702: skimmed MC rfiles
					 ISL/sx-080702/newgenbb: gen BB MC
           vub-combo       152832512     
           moments         191845657
-----------------------------------------------------------------------------

In vub-recoil space the following HBOOK files are around (why??): 
----------------------------------------------------------------------
    AWG23:  ca 50GB in ISL/sx-080702/hbook_gen_mar/ and 
                       ISL/sx-080702/newsig/output/outputdir/

    AWG18:  ca 12GB in ISL/sx-080702/newsignal/output/outputdir/ and  
                       ISL/sx-080702/signalMC/FIXED/
----------------------------------------------------------------------


There is  a minor problem  in that many  of the files  and directories
belong to Alessio,  and his account is gone. Will have  to send a mail
to SCS.


We also occupy diskspace of group EC: 
 
 shire01>du -ks /nfs/farm/g/ec/u05/users/*
 2126409         /nfs/farm/g/ec/u05/users/asarti
 509917864       /nfs/farm/g/ec/u05/users/henning

vub-recoil does not  use this diskspace, I think  (I found no symlinks
to this disk  and nothing in our chains).  The  story is different for
the moments-analysis, I guess.


-> There  are hbook files  to be  deleted, and  some logfiles  are not
   gzip'ed. We can free up >50 GB on relatively short notice, I guess.



2. CPU and Disk Usage for old production
========================================

I  assume that  we dump  identical  ntuples (root  format) as  before.

 o ca. 25% of the generic BB allevents make it into the ntuples. 
 o Without tagbits, the jobs are CPU limited: 

        barb: 1.4sec CPU/allEvent
        noma: 0.7sec CPU/allEvent
   
   The wall clock  time is only marginally (<5%  .. 10%) higher. We'll
   have to see how this is with a CM2 executable on OBJY.

 o  Running with  tagbits  was mostly  relevant  in the  old (K0S  bug
   affected) MC. I  cannot find tcl, log and  rootfiles for this case.
   Since  the jobs  are CPU  limited I  assume that  the time  gain is
   proportional to the events skipped.

   The skim  fractions are 20% (somewhat  less than what  I see above,
   but this may be due to updated(?) purity tables), see the full list
   in http://www.slac.stanford.edu/BFROOT/www/Physics/Analysis/AWG/EHBDOC/skims/allskims.html

 o Diskspace: 7.5 kB/tupleEvent


The above was generic BB MC. Now for data: 

 o CPU:       3.6sec CPU/filterEvent, 0.16sec CPU/allEvent, 
 o Disk:      4.5 kB/tupleEvent



3. Projection for new production
================================

Data: assume 150/fb skimmed CM2
-------------------------------

There are 1.5e7allEvents/fb-1 (see http://www.slac.stanford.edu/BFROOT/www/Computing/DataQuality/SkimmedData.html)

This implies for the semiExcl skim (with a rate of 4% on data)

  -> 150 * 1.5e7 * 0.04  = 9.0e7 events
  -> 9e7*4.5kB = 4.1e8kB = 410 GB

Assuming 1sec CPU/event, this is 25000 CPU hours. (This assumption may
be somewhat optimistic with CM2 ...)  This is what a user can get from
the compute farm in about ONE month (if "run" is running 24/7).


generic BB MC: first assume 400/fb unskimmed SP5 OBJY
-----------------------------------------------------

This is ca. 420e6 allEvents. Assume 20% with semiExcl BRECO skim. 

  -> 0.2 * 420e6 = 8.4e7 events
  -> 8.4e7*7.5kB = 630GB

Assuming again  1sec CPU/event and  the availability of the  tag bits,
this is 23000 CPU hours. If the "skim rate" is higher (because the tag
bits  are  based on  older  purity  tables(?),  the disk  space  could
increase by up to 25%, to 780GB).

If the tagbits  are not available, the total  time would be 125khours,
doable for a team in one month.


Now add 120/fb of skimmed CM2 SP6
---------------------------------
This is ca 24e6 events for the ntuple

  -> 24e6 * 7.5kB = 180GB

The time to run on this is small, O(6000h).




Top of Message | Previous Page | Permalink

Advanced Options


Options

Log In

Log In

Get Password

Get Password


Search Archives

Search Archives


Subscribe or Unsubscribe

Subscribe or Unsubscribe


Archives

March 2010
December 2009
August 2009
January 2009
November 2008
October 2008
September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
December 2005
November 2005
October 2005
September 2005
August 2005
July 2005
June 2005
May 2005
April 2005
March 2005
February 2005
January 2005
December 2004
November 2004
October 2004
September 2004
August 2004
July 2004
June 2004
May 2004
April 2004
March 2004
February 2004
January 2004
December 2003
November 2003
October 2003
September 2003
August 2003
July 2003
June 2003
May 2003
April 2003
March 2003
February 2003
January 2003
December 2002
November 2002
October 2002
September 2002
August 2002
July 2002
June 2002
May 2002
April 2002
March 2002
February 2002
January 2002
December 2001
November 2001
October 2001

ATOM RSS1 RSS2



LISTSERV.SLAC.STANFORD.EDU

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager

Privacy Notice, Security Notice and Terms of Use