Daniel, > Solutions: > 1. Move subchunk query generation to the worker (protocol changes, > dependency shuffling to promote the query templating code into > common). This doesn't really fix anything by itself, except it > improves dispatch speed. > > 2. Let the master know what subChunks are populated. We can build an > index pretty easily from the objectid index. Now the master can skip > worrying about empty subchunks. (similarly, we can create the > emptychunks file). Everyone wins here, minus the time to create the > index. I think we can create it in-memory on startup and maybe cache > it on disk to avoid reading on startup. > > 3. Have the worker figure out what subchunks are populated. The > worker is really in the right position to do this. Coupled with 1., > it would be comparable to 2. I think in the long term the master should only know about chunks. Subchunking should be entirely a local optimization, with local data storage and structures to facilitate it if necessary. We may want to do something besides subchunking -- trees, for example -- and that will be difficult if the master controls it all. > > We can probably do 2. It's the easiest. But not right now, not today. > > >plus if maybe you could get that to work... > > > >SELECT > > ROUND(scisql_fluxToAbMag(uFlux_PS)-scisql_fluxToAbMag(gFlux_PS), 0) > >AS UG, > > ROUND(scisql_fluxToAbMag(gFlux_PS)-scisql_fluxToAbMag(rFlux_PS), 0) > >AS GR, > > ROUND(scisql_fluxToAbMag(rFlux_PS)-scisql_fluxToAbMag(iFlux_PS), 0) > >AS RI, > > ROUND(scisql_fluxToAbMag(iFlux_PS)-scisql_fluxToAbMag(zFlux_PS), 0) > >AS IZ, > > ROUND(scisql_fluxToAbMag(zFlux_PS)-scisql_fluxToAbMag(yFlux_PS), 0) AS ZY > >FROM Object > >WHERE > > scisql_fluxToAbMag(uFlux_PS)-scisql_fluxToAbMag(gFlux_PS) + > > scisql_fluxToAbMag(gFlux_PS)-scisql_fluxToAbMag(rFlux_PS) + > > scisql_fluxToAbMag(rFlux_PS)-scisql_fluxToAbMag(iFlux_PS) + > > scisql_fluxToAbMag(iFlux_PS)-scisql_fluxToAbMag(zFlux_PS) + > > scisql_fluxToAbMag(zFlux_PS)-scisql_fluxToAbMag(yFlux_PS) < <x>; I don't understand this query. Isn't the WHERE clause mathematically equivalent to just u-y? -- Kian-Tat Lim, LSST Data Management, [log in to unmask] ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the QSERV-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1