 |
 |
My mistake, here's what I meant: .../Source/match/dirColNameObjectId .../Source/match/dirTableObject ... Correct? Le 18/12/2014 22:03, Fabrice Jammes a écrit : > Thanks Serge for these clear explanations, > > In this test I was partitioning SimRefObject but I think it is an > error, I'll remove it. (It seems SQL query coverage should be > extended, all queries on SimRefObject are in FIXME state ;-)) > In my case the dirTable will be Object and the dirColName Source, for > the table Source, isn't it? > > Cheers, > > Le 18/12/2014 21:33, Serge Monkewitz a écrit : >> Well, in testcase 01, SimRefObject and RefSrcMatch are not >> partitioned. So Qserv doesn’t really need to know anything about them >> for queries to work (at least from the query analysis point of view).. >> >> If you did decide to partition SimRefObject, it would be its own >> director table, and would have dirTable = SimRefObject and dirColName >> equal to its PK (simRefObjectId I think it was). >> >> RefSrcMatch is a match table, which can be used to join between 2 >> directors. In this case (I’m not sure where exactly the test case 01 >> data is from) it probably links up SimRefObject with the Source >> table. So it would have ZK metadata that looks roughly like: >> >> .../RefSrcMatch/match1 >> .../RefSrcMatch/match/dirColName1simRefObjectId >> .../RefSrcMatch/match/dirColName2sourceId >> .../RefSrcMatch/match/dirTable1SimRefObject >> .../RefSrcMatch/match/dirTable2Source >> .../RefSrcMatch/match/flagColNameflags >> .../RefSrcMatch/partitioning\N >> .../RefSrcMatch/partitioning/subChunks0 >> >> And, since partitioning match table records works differently from >> other kinds of records (each record is associated with 2 positions >> rather than just 1), you have to invoke sph-partition-matches rather >> than sph-partition on it. I don’t know if Andy’s loader supports that >> yet. >> >> Cheers, >> Serge >> >> >> On Dec 18, 2014, at 6:44 PM, Fabrice Jammes <[log in to unmask] >> <mailto:[log in to unmask]>> wrote: >> >>> -------- Forwarded Message -------- >>> Subject: Re: [QSERV-L] LOADER: support for multi-column PK >>> Date: Thu, 18 Dec 2014 18:43:24 -0800 >>> From: Fabrice Jammes <[log in to unmask]> >>> To: Serge Monkewitz <[log in to unmask]> >>> >>> >>> >>> Ok, what should be dirTable, dirColName for SimRefObject, indeed it >>> doesn't have a link with Object table, but with RefSrcMatch. >>> >>> Thanks, it helps a lot to understand ;-) >>> >>> Fabrice >>> >>> On 12/18/2014 06:33 PM, Serge Monkewitz wrote: >>>> Ah. I checked testcase 01, and none of the queries not marked FIXME >>>> join between Object and the other tables you mention. In fact, >>>> there is only one query that does a join at all (1051_nn.sql >>>> <https://dev.lsstcorp.org/cgit/LSST/DMS/testdata/qserv_testdata.git/tree/datasets/case01/queries/1051_nn.sql>), >>>> and it joins Object against itself. >>>> >>>> So if you aren’t loading metadata from .param files anymore and you >>>> omit dirTable and dirColName, nothing will break. If you were to >>>> run a query that joined say Object and Source, the query analysis >>>> code would complain. >>>> >>>> I didn’t check the second test case. >>>> >>>> >>>> On Dec 18, 2014, at 6:26 PM, Fabrice Jammes >>>> <[log in to unmask] <mailto:[log in to unmask]>> wrote: >>>> >>>>> It seems it is: >>>>> >>>>> Database changed >>>>> mysql> show tables; >>>>> +----------------------------------+ >>>>> | Tables_in_qservTest_case01_qserv | >>>>> +----------------------------------+ >>>>> | Filter | >>>>> | LeapSeconds | >>>>> | Logs | >>>>> | Object | >>>>> | ObjectFullOverlap_6630 | >>>>> | ObjectFullOverlap_6631 | >>>>> | ObjectFullOverlap_6800 | >>>>> | ObjectFullOverlap_6801 | >>>>> | ObjectFullOverlap_6968 | >>>>> | ObjectFullOverlap_6970 | >>>>> | ObjectFullOverlap_6971 | >>>>> | ObjectFullOverlap_7138 | >>>>> | ObjectFullOverlap_7140 | >>>>> | ObjectFullOverlap_7308 | >>>>> | ObjectFullOverlap_7478 | >>>>> | ObjectFullOverlap_7648 | >>>>> | Object_1234567890 | >>>>> | Object_6630 | >>>>> | Object_6631 | >>>>> | Object_6800 | >>>>> | Object_6801 | >>>>> | Object_6968 | >>>>> | Object_6970 | >>>>> | Object_6971 | >>>>> | Object_7138 | >>>>> | Object_7140 | >>>>> | Object_7308 | >>>>> | Object_7310 | >>>>> | Object_7478 | >>>>> | Object_7648 | >>>>> | RefSrcMatch | >>>>> | Science_Ccd_Exposure | >>>>> | SimRefObject | >>>>> | Source | >>>>> | Source_1234567890 | >>>>> | Source_6630 | >>>>> | Source_6631 | >>>>> | Source_6800 | >>>>> | Source_6801 | >>>>> | Source_6968 | >>>>> | Source_6970 | >>>>> | Source_6971 | >>>>> | Source_7138 | >>>>> | Source_7140 | >>>>> | Source_7308 | >>>>> | Source_7310 | >>>>> | Source_7478 | >>>>> | Source_7648 | >>>>> | sdqa_Metric | >>>>> +----------------------------------+ >>>>> 49 rows in set (0,00 sec) >>>>> >>>>> >>>>> On 12/18/2014 06:20 PM, Serge Monkewitz wrote: >>>>>> On Dec 18, 2014, at 6:12 PM, Fabrice Jammes >>>>>> <[log in to unmask] <mailto:[log in to unmask]>> wrote: >>>>>> >>>>>>> DeepSource si a view, shouldn't it be RunDeepSource the director >>>>>>> table? >>>>>> In the Winter2013 release, yes it is. But RunDeepSource contains >>>>>> a bunch of extra columns and uses pipeline column naming >>>>>> conventions, so I just dumped from the view instead. Note I had >>>>>> to do other things to make it work in qserv (IIRC I changed the >>>>>> type of BIT columns to TINYINT). Unless the goal is to duplicate >>>>>> the plain mysql setup from stripe 82 data as closely as possible, >>>>>> let’s not get hung up on these details. >>>>>> >>>>>>> Furthermore, it seems dirTable and dirColName are optional for >>>>>>> now, isn't it? Andy doesn't set it in test case01 and 02 and it >>>>>>> works. >>>>>> If you don’t specify them though, the query analysis code will >>>>>> not let you join between related partitioned tables. Is the data >>>>>> in test case01/02 being partitioned at all? >>>>>> >>>>>> Serge >>>>>> ######################################################################## >>>>>> Use REPLY-ALL to reply to list >>>>>> >>>>>> To unsubscribe from the QSERV-L list, click the following link: >>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1 >>>>> >>>>> >>>> >>> >>> >>> >>> >>> ------------------------------------------------------------------------ >>> >>> Use REPLY-ALL to reply to list >>> >>> To unsubscribe from the QSERV-L list, click the following link: >>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1 >>> >> >> >> ------------------------------------------------------------------------ >> >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the QSERV-L list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1 >> > > > ------------------------------------------------------------------------ > > Use REPLY-ALL to reply to list > > To unsubscribe from the QSERV-L list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1 > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the QSERV-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1