 |
 |
So, we said we'd do 10% of DR1. We need to think carefully how we want to look at DR1, because the full data set with indexes, object_extra etc is 1.7 petabyte. I think a fair and realistic test would be to look at 10% of the core data (Object, Source, ForcedSource, Exposures), exercise some scans and joins, but just forget Object_extra (which we can argue will be less frequently used, and testing with it won't really stress qserv software in any serious way anyway. After all, we always ran with Object and Source only in the past too) That basically is ~27 TB + indexes. (The data sizes for DR1: 38 TB Object 186 TB Source 45 TB ForcedSource) We have ~8 TB on each machine at IN2P3, if I recall, so as we said earlier, ~10 machines would be a minimum to run the test, 25 would be comfortable, 50 would be even better. Are we in position to generate Object, Source and ForcedSource tables? Which data set would we be using to start with? Jacek ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the QSERV-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1