Hi Matevz,
Andreas is right in that you can get that effect with async reading.
Though it can be avoided with a better algorithm which, unfortuantely,
wasn't used. So, yes, set the cache to zero for xrdcp, though I would be
interested in seeing what effect it has on transfer rate with a single
source as well as multiple sources.
Andy
On Tue, 13 Nov 2012, Matevz Tadel wrote:
> Thanks Andy, Andreas!
>
> Andreas told me this is a feature of async reading and that it gets
> sequential if one sets readcachesize to 0.
>
> Funny that I never printed offsets before :)
>
> Cheers,
> Matevz
>
> On 11/13/12 10:04, Andrew Hanushevsky wrote:
>> Hi Matevz,
>>
>> This is a known problem. Yes, it bizzare but it is what it is. We will have
>> to
>> wait for 4.0 when we deploy the new client and the new xrdcp which should
>> solve
>> this problem. I agree, it's a real problem.
>>
>> Andy
>>
>> On Mon, 12 Nov 2012, Matevz Tadel wrote:
>>
>>> Hi,
>>>
>>> I'm testing the monitoring collector extension that is capable of storing
>>> full
>>> access details into a root tree. Running a simple xrdcp I realized that
>>> xrdcp
>>> first copies data at offset of 8MB and only then the beginning of the
>>> file.
>>>
>>> t ~ time since file open
>>> l ~ length of request
>>> o ~ offset
>>>
>>> // file_len=4983779
>>> SXrdIoInfo::Dump() N_reqs = 2, N_vec_read_subreqs = 0
>>> 0 Read t=2 l=483 o=4983296
>>> 1 Read t=4 l=4983296 o=0
>>>
>>> // file_len=9311384
>>> SXrdIoInfo::Dump() N_reqs = 2, N_vec_read_subreqs = 0
>>> 0 Read t=2 l=922776 o=8388608
>>> 1 Read t=4 l=8388608 o=0
>>>
>>> This is even funnier for a larger file (~400MB) ... the first 8MB
>>> (offset=0)
>>> are read after 238MB are already read, see below, entry 60.
>>>
>>> What's the secret here? :)
>>>
>>> Cheers,
>>> Matevz
>>>
>>> // file_len=428002641
>>> SXrdIoInfo::Dump() N_reqs = 102, N_vec_read_subreqs = 0
>>> 0 Read t=5 l=4194304 o=8388608
>>> 1 Read t=10 l=4194304 o=12582912
>>> 2 Read t=20 l=4194304 o=16777216
>>> 3 Read t=35 l=4194304 o=20971520
>>> 4 Read t=45 l=4194304 o=25165824
>>> 5 Read t=55 l=4194304 o=29360128
>>> 6 Read t=65 l=4194304 o=33554432
>>> 7 Read t=75 l=4194304 o=37748736
>>> 8 Read t=85 l=4194304 o=41943040
>>> 9 Read t=95 l=4194304 o=46137344
>>> 10 Read t=105 l=4194304 o=50331648
>>> 11 Read t=115 l=4194304 o=54525952
>>> 12 Read t=130 l=4194304 o=58720256
>>> 13 Read t=140 l=4194304 o=62914560
>>> 14 Read t=155 l=4194304 o=67108864
>>> 15 Read t=165 l=4194304 o=71303168
>>> 16 Read t=170 l=4194304 o=75497472
>>> 17 Read t=180 l=4194304 o=79691776
>>> 18 Read t=190 l=4194304 o=83886080
>>> 19 Read t=205 l=4194304 o=88080384
>>> 20 Read t=215 l=4194304 o=92274688
>>> 21 Read t=225 l=4194304 o=96468992
>>> 22 Read t=240 l=4194304 o=100663296
>>> 23 Read t=255 l=4194304 o=104857600
>>> 24 Read t=260 l=4194304 o=109051904
>>> 25 Read t=265 l=4194304 o=113246208
>>> 26 Read t=275 l=4194304 o=117440512
>>> 27 Read t=285 l=4194304 o=121634816
>>> 28 Read t=295 l=4194304 o=125829120
>>> 29 Read t=305 l=4194304 o=130023424
>>> 30 Read t=315 l=4194304 o=134217728
>>> 31 Read t=325 l=4194304 o=138412032
>>> 32 Read t=335 l=4194304 o=142606336
>>> 33 Read t=345 l=4194304 o=146800640
>>> 34 Read t=360 l=4194304 o=150994944
>>> 35 Read t=370 l=4194304 o=155189248
>>> 36 Read t=375 l=4194304 o=159383552
>>> 37 Read t=385 l=4194304 o=163577856
>>> 38 Read t=390 l=4194304 o=167772160
>>> 39 Read t=400 l=4194304 o=171966464
>>> 40 Read t=405 l=4194304 o=176160768
>>> 41 Read t=420 l=4194304 o=180355072
>>> 42 Read t=425 l=4194304 o=184549376
>>> 43 Read t=435 l=4194304 o=188743680
>>> 44 Read t=445 l=4194304 o=192937984
>>> 45 Read t=455 l=4194304 o=197132288
>>> 46 Read t=465 l=4194304 o=201326592
>>> 47 Read t=475 l=4194304 o=205520896
>>> 48 Read t=485 l=4194304 o=209715200
>>> 49 Read t=495 l=4194304 o=213909504
>>> 50 Read t=505 l=4194304 o=218103808
>>> 51 Read t=515 l=4194304 o=222298112
>>> 52 Read t=525 l=4194304 o=226492416
>>> 53 Read t=535 l=4194304 o=230686720
>>> 54 Read t=540 l=4194304 o=234881024
>>> 55 Read t=550 l=4194304 o=239075328
>>> 56 Read t=560 l=4194304 o=243269632
>>> 57 Read t=575 l=4194304 o=247463936
>>> 58 Read t=580 l=4194304 o=251658240
>>> 59 Read t=590 l=4194304 o=255852544
>>> 60 Read t=605 l=8388608 o=0
>>> 61 Read t=625 l=4194304 o=260046848
>>> 62 Read t=635 l=4194304 o=264241152
>>> 63 Read t=645 l=4194304 o=268435456
>>> 64 Read t=655 l=4194304 o=272629760
>>> 65 Read t=665 l=4194304 o=276824064
>>> 66 Read t=675 l=4194304 o=281018368
>>> 67 Read t=680 l=4194304 o=285212672
>>> 68 Read t=690 l=4194304 o=289406976
>>> 69 Read t=695 l=4194304 o=293601280
>>> 70 Read t=705 l=4194304 o=297795584
>>> 71 Read t=715 l=4194304 o=301989888
>>> 72 Read t=730 l=4194304 o=306184192
>>> 73 Read t=745 l=4194304 o=310378496
>>> 74 Read t=760 l=4194304 o=314572800
>>> 75 Read t=770 l=4194304 o=318767104
>>> 76 Read t=780 l=4194304 o=322961408
>>> 77 Read t=790 l=4194304 o=327155712
>>> 78 Read t=800 l=4194304 o=331350016
>>> 79 Read t=810 l=4194304 o=335544320
>>> 80 Read t=820 l=4194304 o=339738624
>>> 81 Read t=825 l=4194304 o=343932928
>>> 82 Read t=835 l=4194304 o=348127232
>>> 83 Read t=850 l=4194304 o=352321536
>>> 84 Read t=855 l=4194304 o=356515840
>>> 85 Read t=870 l=4194304 o=360710144
>>> 86 Read t=880 l=4194304 o=364904448
>>> 87 Read t=895 l=4194304 o=369098752
>>> 88 Read t=905 l=4194304 o=373293056
>>> 89 Read t=915 l=4194304 o=377487360
>>> 90 Read t=925 l=4194304 o=381681664
>>> 91 Read t=935 l=4194304 o=385875968
>>> 92 Read t=940 l=4194304 o=390070272
>>> 93 Read t=950 l=4194304 o=394264576
>>> 94 Read t=970 l=4194304 o=398458880
>>> 95 Read t=975 l=4194304 o=402653184
>>> 96 Read t=985 l=4194304 o=406847488
>>> 97 Read t=995 l=4194304 o=411041792
>>> 98 Read t=1005 l=4194304 o=415236096
>>> 99 Read t=1010 l=4194304 o=419430400
>>> 100 Read t=1025 l=4194304 o=423624704
>>> 101 Read t=1030 l=183633 o=427819008
>>>
>>> ########################################################################
>>> Use REPLY-ALL to reply to list
>>>
>>> To unsubscribe from the XROOTD-DEV list, click the following link:
>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>>>
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-DEV list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-DEV list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|