The xrdcopy will now see the mismatched size and complain:
https://github.com/xrootd/xrootd/commit/06096208603569c54ce7b43a568fe05e1359a89f
]==> xrdcopy --streams 1
root://f01-140-115-e.gridka.de:1094//atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root
- > /dev/null
[ 99%][=================================================>] [760M/760M]
Run: [ERROR] Received corrupted data
Cheers,
Lukasz
On 31.05.2013 16:04, Lukasz Janyst wrote:
> So it may be the proxy. CCing the xrootd developers list.
>
> Lukasz
>
> On 31.05.2013 15:44, Mol, Xavier (SCC) wrote:
>> Hi Lukasz,
>>
>> from the perspective of dCache, today either a transfer for
>> /atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root
>> (which equals
>> /pnfs/gridka.de/atlas/disk-only/atlasdatadisk/rucio/user/ivukotic/a6/57/group.test.hc.NTUP_SMWZ.root
>> which equals 0000000BDA41933C4E8A8EC629636A4F3624 for dCache)
>> succeeded regularly, or this error occurred:
>>
>> "No connection from Xrootd client after 300 seconds. Giving up."
>>
>> Ciao,
>> Xavier.
>>
>>> -----Original Message-----
>>> From: Lukasz Janyst [mailto:[log in to unmask]]
>>> Sent: Friday, May 31, 2013 3:29 PM
>>> To: Mol, Xavier (SCC)
>>> Cc: Ilija Vukotic; Petzold, Andreas (SCC); Lukasz Janyst;
>>> [log in to unmask]; Lincoln Bryant; Wei Yang;
>>> dcache-
>>> [log in to unmask]
>>> Subject: Re: [dcache-admin] Re: default ReadCacheSize value and other
>>> questions
>>>
>>> Hi Xavier,
>>>
>>> I don't thinks so. I have no experience with dCache and this
>>> kind of
>>> setups, but looking at your configs it's either a problem in the proxy
>>> code (Wei, could you test it somehow?) or in dCache itself.
>>>
>>> Cheers,
>>> Lukasz
>>>
>>> On 31.05.2013 15:07, Mol, Xavier (SCC) wrote:
>>>> Hi Lukasz,
>>>>
>>>> this setup is following the instructions for beeing a proxy:
>>>> https://twiki.cern.ch/twiki/bin/view/Atlas/FAXDECloud. As far as I
>>>> know,
>>> we only deviated from that guide with regards to the GSI security
>>> configuration - we tried to grant dteam users (read) access
>>> additionally to ATLAS clients
>>> (https://twiki.cern.ch/twiki/bin/view/Atlas/XrootdVOMSsecPlugin).
>>> This should be unrelated to your
>>> issues, right?
>>>>
>>>> Ciao,
>>>> Xavier.
>>>>
>>>>> -----Original Message-----
>>>>> From: Lukasz Janyst [mailto:[log in to unmask]]
>>>>> Sent: Friday, May 31, 2013 2:40 PM
>>>>> To: Ilija Vukotic
>>>>> Cc: Petzold, Andreas (SCC); Lukasz Janyst;
>>>>> [log in to unmask]; Lincoln Bryant; Wei Yang;
>>>>> dcache-
>>>>> [log in to unmask]
>>>>> Subject: [dcache-admin] Re: default ReadCacheSize value and other
>>>>> questions
>>>>>
>>>>> Hi Andreas,
>>>>>
>>>>> The server behaves incorrectly and erratically when presented
>>>>> with a
>>>>> request for a chunk that exceeds the file boundary. Also, it sometimes
>>>>> refuses to open a file that it has stated correctly. Please see the
>>>>> attached test case. What kind of setup are you running on that box? Is
>>>>> it a proxy, standard xrootd, something else?
>>>>>
>>>>> With the attached test program, I get the following on lxplus:
>>>>>
>>>>> ]==> ./a.out
>>>>> File size: 797152257
>>>>> Read 16777216 bytes at offset 788529152
>>>>> Unable to read: [ERROR] Server responded with an error: [3011]
>>>>> Unable to
>>>>> read
>>>>> /atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root;
>>>>>
>>>>> no such file or directory
>>>>>
>>>>> ]==> ./a.out
>>>>> File size: 797152257
>>>>> Read 16777216 bytes at offset 788529152
>>>>> Status of all operations: OK
>>>>> Received bytes: 8388608
>>>>> Should have received: 8623105
>>>>>
>>>>> ]==> ./a.out
>>>>> File size: 797152257
>>>>> Unable to open: [ERROR] Server responded with an error: [3005]
>>>>> Unable to
>>>>> open
>>>>> /atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root;
>>>>>
>>>>> operation canceled
>>>>>
>>>>> In the case where read returned the OK response but incorrect
>>>>> chunk,
>>>>> xrdcopy should have been able to detect the error. I will make sure
>>>>> it does.
>>>>>
>>>>> Cheers,
>>>>> Lukasz
>>>>>
>>>>>
>>>>> On 30.05.2013 20:35, Ilija Vukotic wrote:
>>>>>> Hi Andreas,
>>>>>>
>>>>>> would you mind trying the same command from lxplus?
>>>>>>
>>>>>> Ilija
>>>>>>
>>>>>> --
>>>>>> Ilija Vukotic • Skype ivukotic • +1 872 230 6435 • University of
>>>>>> Chicago
>>>>>>
>>>>>>
>>>>>>
>>>>>> On May 30, 2013, at 13:32 , Andreas Petzold <[log in to unmask]>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Ilija,
>>>>>>>
>>>>>>> yes, in both cases I was using 3.3.2.
>>>>>>>
>>>>>>> Andreas
>>>>>>>
>>>>>>>
>>>>>>> On 05/30/2013 08:25 PM, Ilija Vukotic wrote:
>>>>>>>> Hi,
>>>>>>>> Was the client 3.3.2?
>>>>>>>>
>>>>>>>> ilija
>>>>>>>>
>>>>>>>> Thumb typed
>>>>>>>>
>>>>>>>> ----- Reply message -----
>>>>>>>> From: "Andreas Petzold" <[log in to unmask]>
>>>>>>>> To: "Ilija Vukotic" <[log in to unmask]>
>>>>>>>> Cc: "Lukasz Janyst" <[log in to unmask]>,
>>>>>>>> "[log in to unmask]"
>>>>>>>> <[log in to unmask]>, "Lincoln Bryant"
>>>>>>>> <[log in to unmask]>, "Wei Yang" <[log in to unmask]>
>>>>>>>> Subject: default ReadCacheSize value and other questions
>>>>>>>> Date: Thu, May 30, 2013 11:54
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Ilja,
>>>>>>>>
>>>>>>>> FYI, I'm able to run xrdcopy successfully for your file on my
>>>>>>>> desktop at
>>>>>>>> KIT and on lxplus.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Andreas
>>>>>>>>
>>>>>>>> On 05/30/2013 06:04 PM, Ilija Vukotic wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I add Andreas and Guenter. Lincoln attempted to xrdcopy file
>>>>>>>>> form FZK today between 17:00 and 18:00 (your time) and failed.
>>>>> We would like to know what is in the servers log. Would you mind
>>>>> looking it up?
>>>>>>>>> (ordinary xrdcp works ok, but xrdcopy always fails from your
>>>>>>>>> site.)
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Ilija
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On May 30, 2013, at 10:57 , Lukasz Janyst <[log in to unmask]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Yes, this is a message that the client gets from the server.
>>>>>>>>>> Can you actually check what is happening server-side?
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Lukasz
>>>>>>>>>>
>>>>>>>>>> On 30.05.2013 17:55, Lincoln Bryant wrote:
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> I also get this error.
>>>>>>>>>>>
>>>>>>>>>>> [lincolnb@uct2-int ~]$ xrdcopy --streams 1 root://f01-140-115-
>>>>> e.gridka.de:1094//atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root
>>>>> - > /dev/null
>>>>>>>>>>> [ 98%][=================================================>]
>>>>>>>>>>> [752M/760M]
>>>>>>>>>>> Run: [ERROR] Server responded with an error: [3011] Unable to
>>>>>>>>>>> read
>>>>> /atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root;
>>>>> no such file or directory
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [lincolnb@uct2-int ~]$ rpm -q xrootd-client
>>>>>>>>>>> xrootd-client-3.3.2-1.el5
>>>>>>>>>>>
>>>>>>>>>>> --Lincoln
>>>>>>>>>>>
>>>>>>>>>>> On May 23, 2013, at 9:36 AM, Ilija Vukotic wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I added Wei and Lincoln to this mail so they can try too.
>>>>>>>>>>>>
>>>>>>>>>>>> I've just retried with both xrdcp and xrdcopy in a virgin
>>>>>>>>>>>> shell on lxplus and still get the same behavior:
>>>>>>>>>>>>
>>>>>>>>>>>> ~ >xrdcopy --streams 1 root://f01-140-115-
>>>>> e.gridka.de:1094//atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root
>>>>> - > /dev/null
>>>>>>>>>>>> [ 98%][=================================================>]
>>>>>>>>>>>> [752M/760M]
>>>>>>>>>>>> Run: [ERROR] Server responded with an error: [3011] Unable
>>>>>>>>>>>> to read
>>>>> /atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root;
>>>>> no such file or directory
>>>>>>>>>>>>
>>>>>>>>>>>> and
>>>>>>>>>>>>
>>>>>>>>>>>> xrdcp --streams 1 root://f01-140-115-
>>>>> e.gridka.de:1094//atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root
>>>>> - > /dev/null
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>> Ilija
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Nullius in verba
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On May 23, 2013, at 2:07 , Lukasz Janyst <[log in to unmask]>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I get this error with both clients.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Lukasz
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 23.05.2013 09:05, Lukasz Janyst wrote:
>>>>>>>>>>>>>> Hi Ilija,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I constantly get:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ]==> xrdcopy --streams 1
>>>>>>>>>>>>>> root://f01-140-115-
>>> e.gridka.de:1094//atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root
>>>
>>>>>>>>>>>>>> - > /dev/null
>>>>>>>>>>>>>> [100%][==================================================]
>>>>>>>>>>>>>> [0/0]
>>>>>>>>>>>>>> Run: [ERROR] Server responded with an error: [3005] Unable
>>>>>>>>>>>>>> to open
>>>>>>>>>>>>>> /atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root;
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> operation canceled
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Are you sure the server config is OK?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Lukasz
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 23.05.2013 02:20, Ilija Vukotic wrote:
>>>>>>>>>>>>>>> Hi Lukasz,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> when doing this:
>>>>>>>>>>>>>>> xrdcopy --streams 1
>>>>>>>>>>>>>>> root://f01-140-115-
>>> e.gridka.de:1094//atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root
>>>
>>>>>>>>>>>>>>> - > /dev/null
>>>>>>>>>>>>>>> from lxplus it always (!) fails between 94% and 98% in
>>>>>>>>>>>>>>> one of the two
>>>>>>>>>>>>>>> ways.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [ 96%][================================================>
>>>>>>>>>>>>>>> ] [736M/760M]
>>>>>>>>>>>>>>> Run: [ERROR] Server responded with an error: [3011]
>>>>>>>>>>>>>>> Unable to read
>>>>>>>>>>>>>>> /atlas/dq2/user/ilijav/HCtest/user.ilijav.HCtest.1/group.test.hc.NTUP_SMWZ.root;
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> no such file or directory
>>>>>>>>>>>>>>> [
>>>>>>>>>>>>>>> 98%][=================================================>]
>>>>>>>>>>>>>>> [748M/760M]
>>>>>>>>>>>>>>> Run: [ERROR] Internal error
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at the same time standard xrdcp works as expected.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Security is off at that server so please feel free to try
>>>>>>>>>>>>>>> and replicate.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>> Ilija
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Nullius in verba
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On May 22, 2013, at 10:47 , Ilija Vukotic
>>>>>>>>>>>>>>> <[log in to unmask]> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I was using standard xrdcp you get at lxplus (which is
>>>>>>>>>>>>>>>> 3.3.2)
>>>>>>>>>>>>>>>> Now I'll repeat it all with xrdcopy. Will let you know
>>>>>>>>>>>>>>>> what I measure.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> Ilija
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Nullius in verba
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On May 22, 2013, at 8:32 , Lukasz Janyst <[log in to unmask]>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Andreas has now done some testing as well:
>>>>>>>>>>>>>>>>> https://github.com/xrootd/xrootd/issues/20
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Lukasz
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 22.05.2013 14:34, Lukasz Janyst wrote:
>>>>>>>>>>>>>>>>>> Hi Ilija,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> which version of xrootd are you using? Would it be
>>>>>>>>>>>>>>>>>> possible for you
>>>>>>>>>>>>>>>>>> to try the new client (xrdcopy) instead of the old one
>>>>>>>>>>>>>>>>>> (xrdcp)? At
>>>>>>>>>>>>>>>>>> least
>>>>>>>>>>>>>>>>>> from version 3.3.2? We were able to get around 750MB/s
>>>>>>>>>>>>>>>>>> with just one
>>>>>>>>>>>>>>>>>> client on 10GB link.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The default for the old client is indeed one
>>>>>>>>>>>>>>>>>> stream, but if you
>>>>>>>>>>>>>>>>>> don't disable the cache completely it will send many
>>>>>>>>>>>>>>>>>> requests
>>>>>>>>>>>>>>>>>> synchronously.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>> Lukasz
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 21.05.2013 09:00, Ilija Vukotic wrote:
>>>>>>>>>>>>>>>>>>> Hi Lukasz,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I was trying to find out why xrdcp is not using the
>>>>>>>>>>>>>>>>>>> full available
>>>>>>>>>>>>>>>>>>> bandwidth at one 10Gb WAN link and so was playing
>>>>>>>>>>>>>>>>>>> with parameters
>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>> number of streams, size of ReadCache etc.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> What I don't understand is that whatever value I set for
>>>>>>>>>>>>>>>>>>> ReadCacheSize I
>>>>>>>>>>>>>>>>>>> get performance roughly 10 times worse than if I set
>>>>>>>>>>>>>>>>>>> nothing.
>>>>>>>>>>>>>>>>>>> Strange thing here is that performance is 10x worse
>>>>>>>>>>>>>>>>>>> even when I
>>>>>>>>>>>>>>>>>>> set it
>>>>>>>>>>>>>>>>>>> to it's default value: 4000000.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Also one more thing is puzzling to me: I have a way
>>>>>>>>>>>>>>>>>>> to intercept
>>>>>>>>>>>>>>>>>>> messages exchanged between xrootd server and client.
>>>>>>>>>>>>>>>>>>> Looking in
>>>>>>>>>>>>>>>>>>> them I
>>>>>>>>>>>>>>>>>>> see that default behavior of xrdcp is to at the start
>>>>>>>>>>>>>>>>>>> request 20
>>>>>>>>>>>>>>>>>>> chunks
>>>>>>>>>>>>>>>>>>> of 4MB each. Documentation claims that the default is
>>>>>>>>>>>>>>>>>>> one stream?
>>>>>>>>>>>>>>>>>>> Could you explain this to me or point me to some
>>>>>>>>>>>>>>>>>>> documentation I
>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>> read?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thanks a lot,
>>>>>>>>>>>>>>>>>>> Ilija
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Dr Ilija Vukotic [log in to unmask]
>>>>>>>>>>>>>>>>>>> University of Chicagohttp://www.vukotic.me
>>>>>>>>>>>>>>>>>>> 5620 S Ellis Ave Tel: +1-773-702-7475
>>>>>>>>>>>>>>>>>>> Chicago IL 60637, USA
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Karlsruhe Institute of Technology (KIT)
>>>>>>>> Steinbuch Centre for Computing (SCC)
>>>>>>>>
>>>>>>>> Andreas Petzold
>>>>>>>>
>>>>>>>> Hermann-von-Helmholtz-Platz 1, Building 449, Room 202
>>>>>>>> D-76344 Eggenstein-Leopoldshafen
>>>>>>>>
>>>>>>>> Tel: +49 721 608 24916
>>>>>>>> Fax: +49 721 608 24972
>>>>>>>> Email: [log in to unmask]
>>>>>>>> www.scc.kit.edu <http://www.scc.kit.edu>
>>>>>>>>
>>>>>>>> KIT – University of the State of Baden-Wuerttemberg and
>>>>>>>> National Research Center of the Helmholtz Association
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Karlsruhe Institute of Technology (KIT)
>>>>>>> Steinbuch Centre for Computing (SCC)
>>>>>>>
>>>>>>> Andreas Petzold
>>>>>>>
>>>>>>> Hermann-von-Helmholtz-Platz 1, Building 449, Room 202
>>>>>>> D-76344 Eggenstein-Leopoldshafen
>>>>>>>
>>>>>>> Tel: +49 721 608 24916
>>>>>>> Fax: +49 721 608 24972
>>>>>>> Email: [log in to unmask]
>>>>>>> www.scc.kit.edu
>>>>>>>
>>>>>>> KIT – University of the State of Baden-Wuerttemberg and
>>>>>>> National Research Center of the Helmholtz Association
>>>>>>>
>>>>>>
>>>>
>>
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|
