 |
 |
Mario, > > Here's something for you and the team to think about in November: how > >would you modify qserv to download and cache chunks on-demand? > > > > Imagine the following scenario: scientist M at university H has access > >to 200+ nodes w. petabytes of storage. The same exact number of nodes and amount of storage as we have? Or something smaller? Are you anticipating that many users will *not* do full-table scans? Once they do, they have everything (for that table). Chunks are expected to be multiple terabytes in size, which means that downloads are hours long. > >It may also solve our issue with the number of replicas needed to > >guard against failure, since we could configure our Archive center > >database to fetch any chunks that it doesn't have (e.g., because the > >nodes have failed) from the Chilean or French site. It's not clear that it's faster to get from France or Chile than from a local backup. In any case, copying from anywhere else still means that we're down. -- Kian-Tat Lim, LSST Data Management, [log in to unmask] ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the QSERV-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1