There are some issues that come up with database tables that are
expected to be used in shared scans. It would be good to have fields in
CSS that indicate if a table should be locked in memory for a shared
scan and its priority/speed of scan. We only want to have a few high use
tables getting locked in memory when they are needed so that they are
only loaded once and used by multiple queries. The only tables currently
expected to get locked in memory are Object, Source, Object_extra, and
Forced but this will probably change and I'd like to build the code so
that it does this automatically off of information in CSS.
The priority/speed of a table is an indication of how long a query using
that table will take and how the table is used.
- Object, while being a fairly large table is considered fast/high
priority. It can often be used in a query by itself.
- Source and ForcedSource are fairly large and usually are joined with
Object, so they are medium speed/priority
- Object_extra is expected to be quite large and need to join with
Object, so it is slow low priority.
The point being the speed/priority of a table should be in CSS so that
the czar can decide during analysis if a query should be fast (finish in
about an hour), medium (about 6 hours), or slow (12 hours). This also
means that code will need to be added to the czar to do this and pass
the information to the worker.
Any thoughts on the best way to do this? Other concerns?
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
|