So, we said we'd do 10% of DR1. We need to think carefully
how we want to look at DR1, because the full data set with
indexes, object_extra etc is 1.7 petabyte.

I think a fair and realistic test would be to look at 10%
of the core data (Object, Source, ForcedSource, Exposures),
exercise some scans and joins, but just forget Object_extra
(which we can argue will be less frequently used, and testing
with it won't really stress qserv software in any serious
way anyway. After all, we always ran with Object and Source
only in the past too)

That basically is ~27 TB + indexes. (The data sizes for DR1:
   38 TB Object
  186 TB Source
   45 TB ForcedSource)


We have ~8 TB on each machine at IN2P3, if I recall,
so as we said earlier, ~10 machines would be a minimum
to run the test, 25 would be comfortable, 50 would be
even better.

Are we in position to generate Object, Source
and ForcedSource tables? Which data set would we be
using to start with?

Jacek

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1