 |
 |
Yes, we call the central objectId-to-chunk index the secondary index. K.T. idea is interesting because it would avoid to run the partitioner on the Source tables. We would only have to read Source table lines, get their objectId field, query the secondary index to get their chunk w.r.t objectId, and then send the lines to the chunk table/file. Look thrifty. On 02/26/2015 01:45 PM, Kian-Tat Lim wrote: > Daniel, > >> The general case is very expensive (lookup position and chunk for each >> position!?), and we are only going to get away with it because our >> bulk-loads for ForcedSource will be spatially-restricted. > I'm pretty sure that ForcedSource and (final) Object tables will > be available at the same time, so the partitioner could look up the > coordinates based on objectId using a simple merge. I thought you were > already going to have a central objectId-to-chunk (or even subchunk?) > index anyway; scanning an input ForcedSource table for all its objectIds > and then doing a single query (or at least batching objectIds to reduce > the number of queries by an order of magnitude) to get the mappings > doesn't sound ridiculous. > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the QSERV-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1