Wait, why is it faster in parallel? Same pipe, right? Unless you are
thinking disjoint sets of source-pipe-dest.
-Daniel
On 09/24/2013 04:44 PM, Jacek Becla wrote:
> As we just talked, my numbers are for data chunks,
> index is up to 2x larger, so we can use 2x larger
> numbers. Data+index come in separate files, so
> they can be transferred in parallel, so I think
> it'd be unfair to assume 3x my numbers though
>
> Jacek
>
>
>
> On 9/24/2013 3:07 PM, Jacek Becla wrote:
>>> Chunks are expected to be multiple terabytes in size, which
>>> means that downloads are hours long.
>>
>> K-T,
>>
>> Based on the baseline, which assumes flat 20K chunks per tables,
>> the largest chunk will be 255 GB. The numbers are (in GB,
>> DR1 --> DR11)
>> - Object: 2 --> 4
>> - ObjExtra: 25 --> 69
>> - Source: 9 --> 255
>> - ForcedSrc: 2 --> 98
>>
>> This is in LDM-141, dbL2, L141 (and nearby)
>>
>> And, that is before compression.
>>
>> We talked about keeping chunk size const rather than #chunks
>> constants, which will probably make us go with DR1-size chunk
>> sizes, thus keeping chunk size closer to 25 GB than 1/4 TB)
>>
>> Jacek
>>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the QSERV-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
|