Hi,
So I don’t know what the loader does with “id”. But:
1.) The partitioner currently does nothing with it - all it cares about are positions.
2.) The duplicator reads 2 config values:
- “id”, the primary key of the table, _NOT_ the name of the director key column.
- “part.id”, which is the PK of the director. This is the director key column.
If you are duplicating a director table, these should be set to the same value, or one should be omitted. Anyway, why does the duplicator care? Recall that when it copies around the input data, it has to rewrite primary keys to avoid duplicate values. Furthermore, it has to rewrite corresponding foreign key values in associated tables so that table relationships are maintained.
Having a compound (or non-integer valued) primary key means that you will not be able to run the duplicator on that table. But I’m not sure why it would hold up loading of data for a test case.
Hi,
I cc to qserv-l list in case Serge could also help. Please read this ea;il form down to up.
Additional information, columns below are referenced in RefSrcMatch, not in Object, and if I look at the dirTable:
fjammes@clrlsst-dbmaster-vm:~/src/qserv_testdata/datasets/case01/data (u/salnikov/DM-621 $) $ grep dirTable *
Object.params:dirTable: Object
SimRefObject.params:dirTable: SimRefObject
Source.params:dirTable: Object
So the id field isn't in the coresponding dirTable for none of these examples.
I don’t get why you are grepping for dirTable (what does this tell you about column names?). Can you explain what you are trying to do?
Cheers,
Serge
Would you have some idea?
Thanks,
On 12/18/2014 05:38 PM, Fabrice Jammes wrote:
[log in to unmask]" type="cite">If you have a look at u/salnikov/DM-621 in qserv_testdata repos, your cfg example file use the primary key of the table they relate to.
Are you sure I can change this, on my side, I really don't know?
fjammes@clrlsst-dbmaster-vm:~/src/qserv_testdata (u/salnikov/DM-621 $%) $ head datasets/case01/data/SimRefObject.cfg
# =====================================
# PT1.2 Object table configuration file
# =====================================
# SimRefObject table primary key column.
id = refObjectId
-----
# Partitioning parameters.
part = {
pos = 'ra, decl'
fjammes@clrlsst-dbmaster-vm:~/src/qserv_testdata (u/salnikov/DM-621 $%) $ head datasets/case01/data/Source.cfg
# =====================================
# PT1.2 Source table configuration file
# =====================================
# Source table primary key column.
id = sourceId
# Position columns other than the partitioning position.
pos = [
'ra, decl'
Cheers,
Fabrice
On 12/18/2014 04:46 PM, Salnikov, Andrei A. wrote:
[log in to unmask]" type="cite">Why do you care about primary key? I don't think 'id' is related
to primary key, it should be a column name in a director table.
Cheers,
Andy
From: Fabrice Jammes [mailto:[log in to unmask]]
Sent: Thursday, December 18, 2014 4:35 PM
To: Salnikov, Andrei A.
Subject: Fwd: [QSERV-L] LOADER: support for multi-column PK
Andy,
After a quick look at the data-loader code, I think you don't manage 'id' as a list.
I' think I need a patch to load data in integration test03.
Thanks,
Fabrice
-------- Forwarded Message --------
Subject: | [QSERV-L] LOADER: support for multi-column PK |
Date: | Thu, 18 Dec 2014 16:21:47 -0800 |
From: | |
To: | |
Hello,
Do you know if I can specify a multi-column primary key in the id field
of a cfg file?
Indeed, for AvgForcedPhotYearly
PRIMARY KEY is (`deepSourceId`,`year`),
Cheers,
Fabrice
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
