Serge,
> If there were, we could just use it for objectId queries as well. This
> is tempting to me, except that I think the partitioner needs something
> like a central “objectId" -> chunk mapping unless we force people to
> supply an associated object position for all partitioned tables.
It doesn't necessarily need to be central -- we could spray the
list of objectIds to the workers and have each check its own chunk(s).
> And if we need it for the partitioner, we might as well take advantage
> on the czar. Once we are there, it does not seem like a big stretch to
> allow using the same sorts of indexes on columns other than the PK of
> the director table.
True.
I think you're right that the per-chunk overhead is always going
to be too big. Too bad, though -- MySQL has indexes down in the
workers, but we can't use them very effectively. On top of that, I
don't think there's much of a way to compress the in-memory
representation of the objectId (or secondary) index. Maybe a Bloom
filter?
--
Kian-Tat Lim, LSST Data Management, [log in to unmask]
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
|