I checked the core files on both machines, it is the same problem. On 08/07/2015 03:48 PM, Jacek Becla wrote: > Andy > > I run 4 simultaneous large queries: 3 object scans and 1 source scan. > Xrootd silently died on 2 machines, no core file. Below I pasted the > tail of the log files. Please start thinking about it while I am trying > to isolate the issue. BTW, it happens towards the end of running. > > Does these log file give you any clue whatsoever? > > Jacek > > > > > ccqserv108 > > 0808 00:22:33.824 [0x7fb739583700] WARN Foreman > (build/wdb/QueryAction.cc:109) - QueryAction overriding dbName with LSST > 0808 00:22:33.824 [0x7fb74a00e700] INFO root > (build/xrdsvc/ChannelStream.cc:122) - returning buffer (256, (more)) > 0808 00:22:33.826 [0x7fb74a00e700] INFO root > (build/xrdsvc/ChannelStream.cc:122) - returning buffer (62, (last)) > 0808 00:22:33.826 [0x7fb74a00e700] INFO root > (build/xrdsvc/SsiSession.cc:153) - RequestFinished type=isStream > 0808 00:22:34.468 [0x7fb739583700] INFO root > (build/wdb/QueryAction.cc:261) - &&& _fillRows size=8 > 0808 00:22:34.469 [0x7fb739583700] DEBUG root > (build/wdb/QueryAction.cc:288) - _transmit last=1 > 0808 00:22:34.469 [0x7fb739583700] DEBUG root > (build/wdb/QueryAction.cc:307) - _transmitHeader > 0808 00:22:34.469 [0x7fb739583700] INFO root > (build/proto/ProtoHeaderWrap.cc:52) - msgBuf size=256 -> [[0]=40, > [1]=13, [2]=2, [3]=0, [4]=0, ..., [251]=48, [252]=48, [253]=48, > [254]=48, [255]=48] > 0808 00:22:34.469 [0x7fb739583700] INFO root > (build/xrdsvc/SsiSession_ReplyChannel.cc:85) - sendStream, checking > stream 0 len=256 last=0 > > ----- > > ccqserv124 > > 0808 00:22:25.329 [0x7f25a67bf700] DEBUG ScanSched > (build/wsched/ChunkDisk.cc:234) - ChunkDisk registering for 2044 : > SELECT MIN(ra) AS QS1_MIN,MAX(ra) AS QS2_MAX,MIN(decl) AS > QS3_MIN,MAX(decl) AS QS4_MAX FROM LSST.Object_2044 AS QST_1_ > p=0x7f257003d0b8 > 0808 00:22:25.329 [0x7f25a67bf700] INFO Foreman > (build/wcontrol/Foreman.cc:296) - Runner running Task: msg: session=7 > chunk=2044 db=LSST entry time=Fri Aug 7 23:52:06 2015 > frag: q=SELECT MIN(ra) AS QS1_MIN,MAX(ra) AS QS2_MAX,MIN(decl) AS > QS3_MIN,MAX(decl) AS QS4_MAX FROM LSST.Object_2044 AS QST_1_, sc= > rt=r_7bff268d0e369dda8fa314132538a96ad_2044_0 > 0808 00:22:25.329 [0x7f25a67bf700] INFO Foreman > (build/wdb/QueryAction.cc:177) - Exec in flight for Db = > q_b762c96f418726ae3457c74c0350d0c4 > 0808 00:22:25.329 [0x7f25a67bf700] WARN Foreman > (build/wdb/QueryAction.cc:109) - QueryAction overriding dbName with LSST > 0808 00:22:25.330 [0x7f25a50b6700] INFO root > (build/xrdsvc/ChannelStream.cc:122) - returning buffer (44, (last)) > 0808 00:22:25.330 [0x7f25a50b6700] INFO root > (build/xrdsvc/SsiSession.cc:153) - RequestFinished type=isStream > 0808 00:22:26.218 [0x7f25a67bf700] INFO root > (build/wdb/QueryAction.cc:261) - &&& _fillRows size=81 > 0808 00:22:26.218 [0x7f25a67bf700] DEBUG root > (build/wdb/QueryAction.cc:288) - _transmit last=1 > 0808 00:22:26.218 [0x7f25a67bf700] DEBUG root > (build/wdb/QueryAction.cc:307) - _transmitHeader > 0808 00:22:26.218 [0x7f25a67bf700] INFO root > (build/proto/ProtoHeaderWrap.cc:52) - msgBuf size=256 -> [[0]=40, > [1]=13, [2]=2, [3]=0, [4]=0, ..., [251]=48, [252]=48, [253]=48, > [254]=48, [255]=48] > 0808 00:22:26.218 [0x7f25a67bf700] INFO root > (build/xrdsvc/SsiSession_ReplyChannel.cc:85) - sendStream, checking > stream 0 len=256 last=0 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the QSERV-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1