[log in to unmask]" type="cite"> Thanks Serge for these clear explanations,
In this test I was partitioning SimRefObject but I think it is an error, I'll remove it. (It seems SQL query coverage should be extended, all queries on SimRefObject are in FIXME state ;-))
In my case the dirTable will be Object and the dirColName Source, for the table Source, isn't it?
Cheers,
Le 18/12/2014 21:33, Serge Monkewitz a écrit :
[log in to unmask]" type="cite">Well, in testcase 01, SimRefObject and RefSrcMatch are not partitioned. So Qserv doesn’t really need to know anything about them for queries to work (at least from the query analysis point of view)..
If you did decide to partition SimRefObject, it would be its own director table, and would have dirTable = SimRefObject and dirColName equal to its PK (simRefObjectId I think it was).
RefSrcMatch is a match table, which can be used to join between 2 directors. In this case (I’m not sure where exactly the test case 01 data is from) it probably links up SimRefObject with the Source table. So it would have ZK metadata that looks roughly like:
.../RefSrcMatch/match 1.../RefSrcMatch/match/dirColName1 simRefObjectId.../RefSrcMatch/match/dirColName2 sourceId.../RefSrcMatch/match/dirTable1 SimRefObject.../RefSrcMatch/match/dirTable2 Source.../RefSrcMatch/match/flagColName flags.../RefSrcMatch/partitioning \N.../RefSrcMatch/partitioning/subChunks 0
And, since partitioning match table records works differently from other kinds of records (each record is associated with 2 positions rather than just 1), you have to invoke sph-partition-matches rather than sph-partition on it. I don’t know if Andy’s loader supports that yet.
Cheers,Serge
On Dec 18, 2014, at 6:44 PM, Fabrice Jammes <[log in to unmask]> wrote:
-------- Forwarded Message --------
Subject: Re: [QSERV-L] LOADER: support for multi-column PK Date: Thu, 18 Dec 2014 18:43:24 -0800 From: Fabrice Jammes <[log in to unmask]> To: Serge Monkewitz <[log in to unmask]>
Ok, what should be dirTable, dirColName for SimRefObject, indeed it doesn't have a link with Object table, but with RefSrcMatch.
Thanks, it helps a lot to understand ;-)
Fabrice
On 12/18/2014 06:33 PM, Serge Monkewitz wrote:
[log in to unmask]" type="cite"> Ah. I checked testcase 01, and none of the queries not marked FIXME join between Object and the other tables you mention. In fact, there is only one query that does a join at all (1051_nn.sql), and it joins Object against itself.
So if you aren’t loading metadata from .param files anymore and you omit dirTable and dirColName, nothing will break. If you were to run a query that joined say Object and Source, the query analysis code would complain.
I didn’t check the second test case.
On Dec 18, 2014, at 6:26 PM, Fabrice Jammes <[log in to unmask]> wrote:
It seems it is:
Database changed
mysql> show tables;
+----------------------------------+
| Tables_in_qservTest_case01_qserv |
+----------------------------------+
| Filter |
| LeapSeconds |
| Logs |
| Object |
| ObjectFullOverlap_6630 |
| ObjectFullOverlap_6631 |
| ObjectFullOverlap_6800 |
| ObjectFullOverlap_6801 |
| ObjectFullOverlap_6968 |
| ObjectFullOverlap_6970 |
| ObjectFullOverlap_6971 |
| ObjectFullOverlap_7138 |
| ObjectFullOverlap_7140 |
| ObjectFullOverlap_7308 |
| ObjectFullOverlap_7478 |
| ObjectFullOverlap_7648 |
| Object_1234567890 |
| Object_6630 |
| Object_6631 |
| Object_6800 |
| Object_6801 |
| Object_6968 |
| Object_6970 |
| Object_6971 |
| Object_7138 |
| Object_7140 |
| Object_7308 |
| Object_7310 |
| Object_7478 |
| Object_7648 |
| RefSrcMatch |
| Science_Ccd_Exposure |
| SimRefObject |
| Source |
| Source_1234567890 |
| Source_6630 |
| Source_6631 |
| Source_6800 |
| Source_6801 |
| Source_6968 |
| Source_6970 |
| Source_6971 |
| Source_7138 |
| Source_7140 |
| Source_7308 |
| Source_7310 |
| Source_7478 |
| Source_7648 |
| sdqa_Metric |
+----------------------------------+
49 rows in set (0,00 sec)
On 12/18/2014 06:20 PM, Serge Monkewitz wrote:
On Dec 18, 2014, at 6:12 PM, Fabrice Jammes <[log in to unmask]> wrote:
DeepSource si a view, shouldn't it be RunDeepSource the director table?In the Winter2013 release, yes it is. But RunDeepSource contains a bunch of extra columns and uses pipeline column naming conventions, so I just dumped from the view instead. Note I had to do other things to make it work in qserv (IIRC I changed the type of BIT columns to TINYINT). Unless the goal is to duplicate the plain mysql setup from stripe 82 data as closely as possible, let’s not get hung up on these details.
Furthermore, it seems dirTable and dirColName are optional for now, isn't it? Andy doesn't set it in test case01 and 02 and it works.If you don’t specify them though, the query analysis code will not let you join between related partitioned tables. Is the data in test case01/02 being partitioned at all?
Serge
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1