Hi,
So I don’t know what the loader does with “id”. But:
1.) The partitioner currently does nothing with it - all it cares about are positions.
2.) The duplicator reads 2 config values:
- “id”, the primary key of the table, _NOT_ the name of the director key column.
- “part.id”, which is the PK of the director. This is the director key column.
If you are duplicating a director table, these should be set to the same value, or one should be omitted. Anyway, why does the duplicator care? Recall that when it copies around the input data, it has to rewrite primary keys to avoid duplicate values. Furthermore, it has to rewrite corresponding foreign key values in associated tables so that table relationships are maintained.
Having a compound (or non-integer valued) primary key means that you will not be able to run the duplicator on that table. But I’m not sure why it would hold up loading of data for a test case.
On Dec 18, 2014, at 5:52 PM, Fabrice Jammes <[log in to unmask]> wrote:
> Hi,
>
> I cc to qserv-l list in case Serge could also help. Please read this ea;il form down to up.
>
> Additional information, columns below are referenced in RefSrcMatch, not in Object, and if I look at the dirTable:
>
> fjammes@clrlsst-dbmaster-vm:~/src/qserv_testdata/datasets/case01/data (u/salnikov/DM-621 $) $ grep dirTable *
> Object.params:dirTable: Object
> SimRefObject.params:dirTable: SimRefObject
> Source.params:dirTable: Object
>
> So the id field isn't in the coresponding dirTable for none of these examples.
I don’t get why you are grepping for dirTable (what does this tell you about column names?). Can you explain what you are trying to do?
Cheers,
Serge
> Would you have some idea?
>
>
>
> Thanks,
>
>
>
> On 12/18/2014 05:38 PM, Fabrice Jammes wrote:
>> If you have a look at u/salnikov/DM-621 in qserv_testdata repos, your cfg example file use the primary key of the table they relate to.
>> Are you sure I can change this, on my side, I really don't know?
>>
>> fjammes@clrlsst-dbmaster-vm:~/src/qserv_testdata (u/salnikov/DM-621 $%) $ head datasets/case01/data/SimRefObject.cfg
>> # =====================================
>> # PT1.2 Object table configuration file
>> # =====================================
>>
>> # SimRefObject table primary key column.
>> id = refObjectId
>>
>> -----
>>
>> # Partitioning parameters.
>> part = {
>> pos = 'ra, decl'
>>
>> fjammes@clrlsst-dbmaster-vm:~/src/qserv_testdata (u/salnikov/DM-621 $%) $ head datasets/case01/data/Source.cfg
>> # =====================================
>> # PT1.2 Source table configuration file
>> # =====================================
>>
>> # Source table primary key column.
>> id = sourceId
>>
>> # Position columns other than the partitioning position.
>> pos = [
>> 'ra, decl'
>>
>> Cheers,
>>
>> Fabrice
>>
>>
>>
>> On 12/18/2014 04:46 PM, Salnikov, Andrei A. wrote:
>>> Why do you care about primary key? I don't think 'id' is related
>>> to primary key, it should be a column name in a director table.
>>>
>>> Cheers,
>>> Andy
>>>
>>>
>>> From: Fabrice Jammes [mailto:[log in to unmask]]
>>> Sent: Thursday, December 18, 2014 4:35 PM
>>> To: Salnikov, Andrei A.
>>> Subject: Fwd: [QSERV-L] LOADER: support for multi-column PK
>>>
>>> Andy,
>>>
>>> After a quick look at the data-loader code, I think you don't manage 'id' as a list.
>>> I' think I need a patch to load data in integration test03.
>>>
>>> Thanks,
>>>
>>> Fabrice
>>>
>>>
>>> -------- Forwarded Message --------
>>> Subject:
>>> [QSERV-L] LOADER: support for multi-column PK
>>> Date:
>>> Thu, 18 Dec 2014 16:21:47 -0800
>>> From:
>>> Fabrice Jammes <[log in to unmask]>
>>> To:
>>> qserv-l <[log in to unmask]>
>>>
>>>
>>> Hello,
>>>
>>> Do you know if I can specify a multi-column primary key in the id field
>>> of a cfg file?
>>> Indeed, for AvgForcedPhotYearly
>>> PRIMARY KEY is (`deepSourceId`,`year`),
>>>
>>> Cheers,
>>>
>>> Fabrice
>>>
>>> ########################################################################
>>> Use REPLY-ALL to reply to list
>>>
>>> To unsubscribe from the QSERV-L list, click the following link:
>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>>>
>>>
>>
>
>
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the QSERV-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
