Daniel,

> Solutions:
> 1. Move subchunk query generation to the worker (protocol changes,
> dependency shuffling to promote the query templating code into
> common). This doesn't really fix anything by itself, except it
> improves dispatch speed.
> 
> 2. Let the master know what subChunks are populated. We can build an
> index pretty easily from the objectid index. Now the master can skip
> worrying about empty subchunks. (similarly, we can create the
> emptychunks file). Everyone wins here, minus the time to create the
> index. I think we can create it in-memory on startup and maybe cache
> it on disk to avoid reading on startup.
> 
> 3. Have the worker figure out what subchunks are populated. The
> worker is really in the right position to do this. Coupled with 1.,
> it would be comparable to 2.

I think in the long term the master should only know about chunks.
Subchunking should be entirely a local optimization, with local data
storage and structures to facilitate it if necessary.  We may want to do
something besides subchunking -- trees, for example -- and that will be
difficult if the master controls it all.

> 
> We can probably do 2. It's the easiest. But not right now, not today.
> 
> >plus if maybe you could get that to work...
> >
> >SELECT
> >    ROUND(scisql_fluxToAbMag(uFlux_PS)-scisql_fluxToAbMag(gFlux_PS), 0)
> >AS UG,
> >    ROUND(scisql_fluxToAbMag(gFlux_PS)-scisql_fluxToAbMag(rFlux_PS), 0)
> >AS GR,
> >    ROUND(scisql_fluxToAbMag(rFlux_PS)-scisql_fluxToAbMag(iFlux_PS), 0)
> >AS RI,
> >    ROUND(scisql_fluxToAbMag(iFlux_PS)-scisql_fluxToAbMag(zFlux_PS), 0)
> >AS IZ,
> >    ROUND(scisql_fluxToAbMag(zFlux_PS)-scisql_fluxToAbMag(yFlux_PS), 0) AS ZY
> >FROM Object
> >WHERE
> >    scisql_fluxToAbMag(uFlux_PS)-scisql_fluxToAbMag(gFlux_PS) +
> >    scisql_fluxToAbMag(gFlux_PS)-scisql_fluxToAbMag(rFlux_PS) +
> >    scisql_fluxToAbMag(rFlux_PS)-scisql_fluxToAbMag(iFlux_PS) +
> >    scisql_fluxToAbMag(iFlux_PS)-scisql_fluxToAbMag(zFlux_PS) +
> >    scisql_fluxToAbMag(zFlux_PS)-scisql_fluxToAbMag(yFlux_PS) < <x>;

I don't understand this query.  Isn't the WHERE clause mathematically
equivalent to just u-y?

-- 
Kian-Tat Lim, LSST Data Management, [log in to unmask]

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1