I will add that per baseline (storage/sizing model), we will have
at the end of the survey:
* 1.1 E09 rows in Raw_Ccd_Exposure
* 5.6 E08 rows in Science_Ccd_Visit
* 9.0 E09 rows in Science_Amp_Visit
So indeed, we are talking about billions, approaching 10 in fact
for Science_Amp*. this is for 2031 though, in DR1 for instance
we will have only 360GB for Raw_Amp_Exposure and 30GB for
Raw_Ccd_Exposure (sizewise_, and that should be easy to cache
on SSD etc.
Jacek
On 09/10/2014 05:43 PM, Daniel L. Wang wrote:
> Some additional notes from the hangout:
>
> Non-partitioned tables:
> * Serge is concerned about the size of these tables that we're planning
> on putting on a shared filesystem or replicating on every worker node.
> He notes that Science_Ccd_Exposure itself will be about 1 billion
> (relatively-wide) rows. 189 ccds * 5.5million images ~= 1 billion.
> If we decide to partition it, we would need the addition of another
> table type in the system (and greatly increase complexity in the
> join-analysis code).
> * Daniel hopes that we can have a wonderfully-cached shared filesystem
> so we won't need such complicated code. Also, this is something that
> won't be a big deal in early DR catalogs.
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the QSERV-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
|