Hi Charlie,
I don't know the answer. HammerCloud is a new thing. I will ask on Wed's US
phone meeting. I do have some guesses:
"mean prepare inputs time" should at least include copying files from
storage to batch nodes. This is likely the longest part. It may also include
querying local file catalogs for file locations in storage. This should be
pretty quick, in seconds. I guess it doesn't include transferring data from
BNL to SLAC, as that will be a measurement of DDM performance, and should be
much longer than 48s. It also shouldn't include time to access remote
condDB.
It is a little surprise that it took hours for other sites to get data to
their batch nodes. But if there are lots of input files, that could be. For
SLAC, we only copy non-ROOT files (like DBRelease.tar.gz) to batch nodes.
ROOT files are read directly from the storage. Again, shorter MPIT doesn't
mean our storage is better.
There is no easy way to tell CPU utilization for a job because there is no
like between LSF IDs and Panda IDs.
We report to US ATLAS a list of CPU types and numbers of cores of each type.
US ATLAS calculates a weighted average based on their SI2K (will use
HEP-SPEC in the future), and use it to count the CPU usage for each jobs.
Wei Yang | [log in to unmask] | 650-926-3338(O)
> From: "Young, Charles C." <[log in to unmask]>
> Date: Thu, 23 Apr 2009 23:48:50 -0700
> To: Wei Yang <[log in to unmask]>, atlas-sccs-planning-l
> <[log in to unmask]>
> Cc: "Young, Charles C." <[log in to unmask]>
> Subject: RE: [Usatlas-prodsys-l] first HammerCloud jobs for US cloud
>
> Hi Wei,
>
> Thanks! Some questions. What is involved in the "mean prepare inputs time"
> step? Is it copying input file from (local?) storage to worker node? Is there
> some preparation of the input data beyond moving it around?
>
> Can we find CPU percent normalized to the execution step only? I.e. exclude
> file copy overhead.
>
> Don't understand definition on page 2. utilization =
> cpuconsumption/cpufactor/(stoptime-starttime). What is cpufactor?
|
