Jacek,

> >>	Chunks are expected to be multiple terabytes in size, which
> >>means that downloads are hours long.

	I was wrong about this.

> >Based on the baseline, which assumes flat 20K chunks per tables,
> >the largest chunk will be 255 GB. The numbers are (in GB,
> >DR1 --> DR11)
> >   - Object:    2 -->   4
> >   - ObjExtra: 25 -->  69
> >   - Source:    9 --> 255
> >   - ForcedSrc: 2 -->  98

	For the fault tolerance use case, we need the total of all
tables, compressed where appropriate, plus the indexes and any other
overhead, multiplied by the number of chunks on each node (including
replicas) divided by the number of nodes.  I think this is where I got
terabytes, as we're going to have around 100 chunks per node plus
another 100 replica chunks, but dividing that by 200 or so nodes gets us
back to each node needing to communicate one chunk's data.  This means
that the vulnerability window is actually only a few minutes at worst,
assuming a 100 Gbit network and 10 Gbyte/sec raw throughput.

	(By the way, do you have overlap in the spreadsheet anywhere?)

-- 
Kian-Tat Lim, LSST Data Management, [log in to unmask]

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1