Some random comments (not saying anything about the things not commented
on):

> >   - automated testing is working, but many queries broken
> Some of those tests will be broken for quite some time. Hence, let's
> make sure the test report shows which tests passed and failed,
> because collapsing lots of passed test plus one failure into a
> single red "FAILED" report is not useful. Maybe this is already
> fixed?

	If they're not going to be fixed, the typical action is to skip
the test and print a warning (or, with a better test harness, print the
number of skipped tests).

> >     - how smart/dumb a worker should be?
> >        - perhaps a little smarter?
> The worker should know as little as possible, but it should know a
> lot about what it owns. This is more than it knows now.
> It should not know about the state of other workers. I would like
> the worker to do zero parsing. If it needs to derive
> characteristics/stats from the query, the master should pass those
> annotations down.

	Sounds good to me.

> >        - What are cons of having smarter worker?
> Coordination in maintaining a consistent state.
> More decisions made by the worker means a more complex system, but
> possibly one that scales better. More decisions by the worker mean
> that worker behavior is more complex to understand. Dumb workers are
> easy to understand and lead to a simpler system.

	The latter is not necessarily true, if the interface to the
worker and the division of responsibilities is clear and logical.

> >     - chunk mapping
> I sort of tried to do this, but the chunking concept needs some
> design. I think K-T finds that all possible qserv partitioning
> schemes will have chunk (and subChunk), but the allocation of
> chunkId would differ.

	Not necessarily subchunk; I can think of ways (but not simple
ones) of doing the self-joins that wouldn't require them.

-- 
Kian-Tat Lim, LSST Data Management, [log in to unmask]

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1