Jacek,

> >	If you don't run manual quality checks after doing a "LOAD DATA
> >INFILE", why should you need to do so after doing a qserv load?
> 
> Where did I imply it? We didn't plan any manual quality checks,
> the only "manual" operations that we thought a person would do
> would be:
>  a) someone responsible for producing the data we are loading
>     needs to come up with configuration (e.g., needs to decide
>     which tables to partition, what are partition parameters etc)

	I agree.  (But I don't necessarily like the mention of
"location" in there; that should be controlled by the system
exclusively.)

>  b) DBA sends a command "publish data set <x>", I think letting
>     the system to automatically publish data after it is all
>     ready would be too much

	What is the DBA doing before sending the command?  Is this just
a matter of timing, or is the DBA supposed to be verifying something?
The latter is what I was worried about.  I'm also worried about the
assumption that we need lots of extra disk space for staging.  For
read-only data, loading chunks into their final locations as directly as
possible (but only enabling their usage in the metadata when at least
one copy of each has been loaded) would not seem to be a problem.

-- 
Kian-Tat Lim, LSST Data Management, [log in to unmask]

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1