URL:
<http://savannah.cern.ch/bugs/?88259>
Summary: seg fault with xrdcp extreme copy
Project: XROOTD
Submitted by: bdouglas
Submitted on: 2011-10-28 16:39
Severity: 3 - Normal
Priority: 5 - Normal
Status: None
Privacy: Public
Assigned to: None
Originator Email:
Open/Closed: Open
Discussion Lock: Any
Fixed by commit(s):
_______________________________________________________
Details:
Yesterday,
I used xprep to trigger frm to copy files with xrdcp extreme copy
from various sites across the ATLAS xrootd federation.
I got a lot of seg faults with xrdcp .
Here is some of the information that I collected -
>From - syslog -
Oct 27 18:10:29 wrk2prv kernel: xrdcp[1912]: segfault at 00000000454d99d0 rip
00000035a5e0bd2d rsp 00007fff72c91360 error 4
Oct 27 18:22:45 wrk2prv kernel: xrdcp[2625]: segfault at 00000000463319d0 rip
00000035a5e0bd2d rsp 00007fff518d9ac0 error 4
Oct 27 18:22:51 wrk2prv kernel: xrdcp[2646]: segfault at 0000000045aa39d0 rip
00000035a5e0bd2d rsp 00007fffac126ae0 error 4
Oct 27 18:28:20 wrk2prv kernel: xrdcp[2912]: segfault at 00000000469fa9d0 rip
00000035a5e0bd2d rsp 00007fff90d716e0 error 4
Oct 27 18:31:02 wrk2prv kernel: xrdcp[3058]: segfault at 0000000044bd29d0 rip
00000035a5e0bd2d rsp 00007fff8af27950 error 4
Oct 27 18:35:25 wrk2prv kernel: xrdcp[3288]: segfault at 0000000044e099d0 rip
00000035a5e0bd2d rsp 00007fff4eb53fa0 error 4
Oct 27 19:05:03 wrk2prv kernel: xrdcp[4731]: segfault at 00000000456de9d0 rip
00000035a5e0bd2d rsp 00007ffff97347e0 error 4
Oct 27 19:10:04 wrk2prv kernel: xrdcp[5083]: segfault at 0000000046d019d0 rip
00000035a5e0bd2d rsp 00007fff17483d90 error 4
Oct 27 19:23:53 wrk2prv kernel: xrdcp[5862]: segfault at 0000000046e2d9d0 rip
00000035a5e0bd2d rsp 00007fffdaee0450 error 4
Oct 27 19:27:15 wrk2prv kernel: xrdcp[6041]: segfault at 0000000046c019d0 rip
00000035a5e0bd2d rsp 00007ffff72801e0 error 4
Oct 27 19:30:54 wrk2prv kernel: xrdcp[6251]: segfault at 00000000456f99d0 rip
00000035a5e0bd2d rsp 00007fffc9dc4f00 error 4
Oct 27 19:36:07 wrk2prv kernel: xrdcp[6461]: segfault at 0000000044ad79d0 rip
00000035a5e0bd2d rsp 00007fff6a833be0 error 4
Oct 27 19:37:37 wrk2prv kernel: xrdcp[6542]: segfault at 0000000045ac19d0 rip
00000035a5e0bd2d rsp 00007fff351a8460 error 4
Oct 27 19:38:22 wrk2prv kernel: xrdcp[6597]: segfault at 00000000472499d0 rip
00000035a5e0bd2d rsp 00007fff9e90f670 error 4
Oct 27 19:40:36 wrk2prv kernel: xrdcp[6719]: segfault at 000000004639e9d0 rip
00000035a5e0bd2d rsp 00007fff6eb67e90 error 4
Oct 27 19:43:11 wrk2prv kernel: xrdcp[6845]: segfault at 00000000472669d0 rip
00000035a5e0bd2d rsp 00007fff9122ce80 error 4
Oct 27 19:45:10 wrk2prv kernel: xrdcp[6933]: segfault at 00000000446399d0 rip
00000035a5e0bd2d rsp 00007fff22607ce0 error 4
Oct 27 19:45:43 wrk2prv kernel: xrdcp[6993]: segfault at 0000000045f029d0 rip
00000035a5e0bd2d rsp 00007fffbd3ec220 error 4
- Note there are many other failures -
>From frm log -
/etc/xrootd/new-stagein.sh: line 63: 4731 Segmentation fault
${bindir}/xrdcp -x -f -s ${rfn} ${tfn} >> $TMPFILE 2>&1
111027 19:05:03 13534 xfr_Run: `/tmp/new-stagein.sh.jK4712' ->
`/local/xrootd/a/atlas/dq2/data11_7TeV/NTUP_TOP/r2276_p516_p523_p530_p577/data11_7TeV.00178109.physics_Egamma.merge.NTUP_TOP.r2276_p516_p523_p530_p577_tid367183_00/NTUP_TOP.367183._000044.root.1.fail'
111027 19:05:03 13534 xfr_Run: removed `/tmp/new-stagein.sh.jK4712'
111027 19:05:03 13534 xfr_Run: /etc/xrootd/new-stagein.sh ended with status
5
>From - log of copy script
failed_file name =
/local/xrootd/a/atlas/dq2/data11_7TeV/NTUP_TOP/r2276_p516_p523_p530_p577/data11_7TeV.00178109.physics_Egamma.merge.NTUP_TOP.r2276_p516_p523_p530_p577_tid367183_00/NTUP_TOP.367183._000044.root.1.fail
lfn:
/atlas/dq2/data11_7TeV/NTUP_TOP/r2276_p516_p523_p530_p577/data11_7TeV.00178109.physics_Egamma.merge.NTUP_TOP.r2276_p516_p523_p530_p577_tid367183_00/NTUP_TOP.367183._000044.root.1
pfn:
/local/xrootd/a/atlas/dq2/data11_7TeV/NTUP_TOP/r2276_p516_p523_p530_p577/data11_7TeV.00178109.physics_Egamma.merge.NTUP_TOP.r2276_p516_p523_p530_p577_tid367183_00/NTUP_TOP.367183._000044.root.1
rfn:
root://glrd.usatlas.org:1094//atlas/dq2/data11_7TeV/NTUP_TOP/r2276_p516_p523_p530_p577/data11_7TeV.00178109.physics_Egamma.merge.NTUP_TOP.r2276_p516_p523_p530_p577_tid367183_00/NTUP_TOP.367183._000044.root.1?tried=+1213headprv.hep.anl.gov
tfn:
root://wrk2prv.hep.anl.gov:1094//atlas/dq2/data11_7TeV/NTUP_TOP/r2276_p516_p523_p530_p577/data11_7TeV.00178109.physics_Egamma.merge.NTUP_TOP.r2276_p516_p523_p530_p577_tid367183_00/NTUP_TOP.367183._000044.root.1
copy command: /usr/bin/xrdcp -x -f -s
root://glrd.usatlas.org:1094//atlas/dq2/data11_7TeV/NTUP_TOP/r2276_p516_p523_p530_p577/data11_7TeV.00178109.physics_Egamma.merge.NTUP_TOP.r2276_p516_p523_p530_p577_tid367183_00/NTUP_TOP.367183._000044.root.1?tried=+1213headprv.hep.anl.gov
root://wrk2prv.hep.anl.gov:1094//atlas/dq2/data11_7TeV/NTUP_TOP/r2276_p516_p523_p530_p577/data11_7TeV.00178109.physics_Egamma.merge.NTUP_TOP.r2276_p516_p523_p530_p577_tid367183_00/NTUP_TOP.367183._000044.root.1
Extreme Copy enabled.
Source #1
root://192.17.18.38:1094//atlas/dq2/data11_7TeV/NTUP_TOP/r2276_p516_p523_p530_p577/data11_7TeV.00178109.physics_Egamma.merge.NTUP_TOP.r2276_p516_p523_p530_p577_tid367183_00/NTUP_TOP.367183._000044.root.1?tried=+1213headprv.hep.anl.gov
Source #2
root://192.17.18.40:1094//atlas/dq2/data11_7TeV/NTUP_TOP/r2276_p516_p523_p530_p577/data11_7TeV.00178109.physics_Egamma.merge.NTUP_TOP.r2276_p516_p523_p530_p577_tid367183_00/NTUP_TOP.367183._000044.root.1?tried=+1213headprv.hep.anl.gov
Source #3
root://128.135.158.186:1094//atlas/dq2/data11_7TeV/NTUP_TOP/r2276_p516_p523_p530_p577/data11_7TeV.00178109.physics_Egamma.merge.NTUP_TOP.r2276_p516_p523_p530_p577_tid367183_00/NTUP_TOP.367183._000044.root.1?tried=+1213headprv.hep.anl.gov
[
_______________________________________________________
Reply to this item at:
<http://savannah.cern.ch/bugs/?88259>
_______________________________________________
Message sent via/by LCG Savannah
http://savannah.cern.ch/
