Print

Print


Since the original email covered a few different points, I'm splitting
it out into separate threads.  This covers issue 3, the "getrusage"
calls.



 >> Question 3)
 >>
 >> I also see from strace output that the code is calling 'getrusage' excessively, does this
 >> really need to be checked 2500 times per second?
 >>
 >> Here's a count of system calls made in 90 seconds.
 >>  232452 getrusage
 >>  77484 gettimeofday
 >>  45394 time
 >>  16750 futex
 >>   15908 timer_settime
 >>   9131 clock_gettime
 >>   8373 send
 >>   8373 poll
 >>   1930 write
 >>   1137 lstat64
 >>    758 read
 >>    758 open
 >>    758 close
 >>     62 rt_sigaction
 >>     31 _llseek

 >    Are you sure that getrusage is called from XRootD and not from some
 > other place in your framework? I could not quite reproduce this issue
 > with xrdcp which uses the same underlying API.

It turns out the 'getrusage' is coming from Gaudi as the following
stack trace shows.  What is the appropriate place to report this?


Breakpoint 1, 0xf7668110 in getrusage () from /lib/libc.so.6
#0  0xf7668110 in getrusage () from /lib/libc.so.6
#1  0xf47f213c in System::ProcessDescriptor::query(long, System::InfoType, System::KERNEL_USER_TIMES*) () from /opt/atlas/software/manageTier3SW/ATLASLocal
RootBase/Athena/i686_slc5_gcc43_opt/16.0.3/GAUDI/v21r9p3a/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiKernel.so
#2  0xf47fb4ff in System::userTime(System::TimeType, System::InfoType, long) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_
gcc43_opt/16.0.3/GAUDI/v21r9p3a/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiKernel.so
#3  0xf479b2dc in ChronoEntity::stop() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/GAUDI/v21r9p3a/Instal
lArea/i686-slc5-gcc43-opt/lib/libGaudiKernel.so
#4  0xf3409c58 in ChronoStatSvc::chronoStop(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /opt/atlas/software/mana
geTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/GAUDI/v21r9p3a/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiSvc.so
#5  0xf12330f4 in AthenaPoolCnvSvc::setObjPtr(void*&, pool::Token const*) const () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_sl
c5_gcc43_opt/16.0.3/AtlasCore/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libAthenaPoolCnvSvcLib.so
#6  0xe624f3c1 in StatusCode T_AthenaPoolCustCnv<TruthParticleContainer, TruthParticleContainer_p6>::poolToObject<TruthParticleContainer_p6>(std::basic_str
ing<char, std::char_traits<char>, std::allocator<char> > const&, TruthParticleContainer_p6*&) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/
Athena/i686_slc5_gcc43_opt/16.0.3/AtlasEvent/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libMcParticleEventAthenaPoolPoolCnv.so
#7  0xe623b649 in TruthParticleContainerCnv::createTransient() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0
.3/AtlasEvent/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libMcParticleEventAthenaPoolPoolCnv.so
#8  0xe62498bc in T_AthenaPoolCustomCnv<TruthParticleContainer, TruthParticleContainer_p6>::PoolToDataObject(DataObject*&, std::basic_string<char, std::cha
r_traits<char>, std::allocator<char> > const&) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/AtlasEvent/16
.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libMcParticleEventAthenaPoolPoolCnv.so
#9  0xf1253342 in AthenaPoolConverter::createObj(IOpaqueAddress*, DataObject*&) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_sl
c5_gcc43_opt/16.0.3/AtlasCore/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libAthenaPoolCnvSvcLib.so
#10 0xf2c24506 in AthCnvSvc::makeCall(int, bool, bool, bool, IOpaqueAddress*&, DataObject*&) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/A
thena/i686_slc5_gcc43_opt/16.0.3/AtlasCore/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libAthenaBaseComps.so
#11 0xf2c24bbe in AthCnvSvc::createObj(IOpaqueAddress*, DataObject*&) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_o
pt/16.0.3/AtlasCore/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libAthenaBaseComps.so
#12 0xf123d085 in AthenaPoolCnvSvc::createObj(IOpaqueAddress*, DataObject*&) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_
gcc43_opt/16.0.3/AtlasCore/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libAthenaPoolCnvSvcLib.so
#13 0xf358b9e0 in PersistencySvc::makeCall(int, IOpaqueAddress*&, DataObject*&) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_sl
c5_gcc43_opt/16.0.3/GAUDI/v21r9p3a/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiSvc.so
#14 0xf358bd4f in PersistencySvc::createObj(IOpaqueAddress*, DataObject*&) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gc
c43_opt/16.0.3/GAUDI/v21r9p3a/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiSvc.so
#15 0xf2b89f28 in SG::DataProxy::accessData() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/AtlasCore/16.0
.3/InstallArea/i686-slc5-gcc43-opt/lib/libSGTools.so
#16 0xe7611731 in TruthParticleContainer* SG::DataProxy_cast<TruthParticleContainer>(SG::DataProxy*) () from /scratch/Panda_Pilot_12278_1305546093/PandaJob_1237132242_1305546198/workDir/InstallArea/i686-slc5-gcc43-opt/lib/libUserAnalysis.so
#17 0xe761c8cb in StatusCode StoreGateSvc::retrieve<TruthParticleContainer, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >(TruthParticleContainer const*&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /scratch/Panda_Pilot_12278_1305546093/PandaJob_1237132242_1305546198/workDir/InstallArea/i686-slc5-gcc43-opt/lib/libUserAnalysis.so
#18 0xe75d0b5a in AnalysisSkeleton::dumpMuonTrigger() () from /scratch/Panda_Pilot_12278_1305546093/PandaJob_1237132242_1305546198/workDir/InstallArea/i686-slc5-gcc43-opt/lib/libUserAnalysis.so
#19 0xe760085f in AnalysisSkeleton::CBNT_execute() () from /scratch/Panda_Pilot_12278_1305546093/PandaJob_1237132242_1305546198/workDir/InstallArea/i686-slc5-gcc43-opt/lib/libUserAnalysis.so
#20 0xe7b4c1f5 in CBNT_AthenaAwareBase::execute() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/AtlasEvent/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libCBNT_UtilsLib.so
#21 0xf47c251d in Algorithm::sysExecute() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/GAUDI/v21r9p3a/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiKernel.so
22 0xe7998e9b in CBNT_AthenaAware::execute() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/AtlasEvent/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libCBNT_Athena.so
#23 0xf47c251d in Algorithm::sysExecute() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/GAUDI/v21r9p3a/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiKernel.so
#24 0xe7e54203 in AthSequencer::execute() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/AtlasCore/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiSequencer.so
#25 0xf47c251d in Algorithm::sysExecute() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/GAUDI/v21r9p3a/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiKernel.so
#26 0xe7e54203 in AthSequencer::execute() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/AtlasCore/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiSequencer.so
#27 0xf47c251d in Algorithm::sysExecute() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/GAUDI/v21r9p3a/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiKernel.so
#28 0xf2d1464f in AthenaEventLoopMgr::executeAlgorithms() () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/AtlasCore/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libAthenaServices.so
#29 0xf2d1ae3e in AthenaEventLoopMgr::executeEvent(void*) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/AtlasCore/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libAthenaServices.so
#30 0xf2d19b22 in AthenaEventLoopMgr::nextEvent(int) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/AtlasCore/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libAthenaServices.so
#31 0xf2d137db in AthenaEventLoopMgr::executeRun(int) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/AtlasCore/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libAthenaServices.so
#32 0xf33cea49 in ApplicationMgr::executeRun(int) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/GAUDI/v21r9p3a/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiSvc.so
#33 0xf3d4e94d in (anonymous namespace)::method_4267(void*, void*, std::vector<void*, std::allocator<void*> > const&, void*) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/GAUDI/v21r9p3a/InstallArea/i686-slc5-gcc43-opt/lib/libGaudiKernelDict.so
#34 0xf528c437 in ROOT::Cintex::Method_stub_with_context(ROOT::Cintex::StubContext_t*, G__value*, char const*, G__param*, int) () from /opt/atlas/software/manageTier3SW/ATLASLocalRootBase/Athena/i686_slc5_gcc43_opt/16.0.3/DetCommon/16.0.3/InstallArea/i686-slc5-gcc43-opt/lib/libCintex.so
#35 0xf30d7034 in ?? ()
#36 0x0eb0a280 in ?? ()
#37 0x0fb9c004 in ?? ()
#38 0x00000000 in ?? ()