LISTSERV 16.5 - QSERV-L Archives

Yes, that is what my loading script had been doing.  On those
tables, drop the primary key, and add an index on the objectId.

Douglas


On 05/20/2013 11:23 AM, Becla, Jacek wrote:
> How about changing the index on objectId for Object_Overlap
> tables (only!) from primary key to a non unique key, something
> like:
>
> alter table Object_Overlap drop primary key, add key (objectId);
>
> Jacek
>
>
> On 05/17/2013 10:45 AM, Daniel L. Wang wrote:
>> The other thing to remember is that objectId is not a primary key in
>> Object_*Overlap* tables. Recall that the concept of an overlap allows
>> "duplicate" rows that differ only in subChunkId. This allows
>> simpler/faster on-the-fly creation of subchunk tables.
>>
>> We haven't thought about how this pseudo-duplication affects schema
>> handling yet (because we have put very little thought into schema
>> handling for non baseline LSST schema).
>>
>> -Daniel
>>
>> On 05/17/2013 10:30 AM, Douglas Smith wrote:
>>> I thought we had that fixed in the latest versions?  But we decided in
>>> the past that for duplicated data for testing we aren't concerned about
>>> this.  That we just need to serve a large amount of data.  And you can
>>> just add "IGNORE" to the insert statement.
>>>
>>> Oh, and also, this is the overlap tables.  Yes, on the overlap tables, this
>>> will be true, and you need to remove the primary key from the table
>>> for the overlap tables.  And we only need the overlap tables for the
>>> Object table, not for the source table.
>>>
>>> Douglas
>>>
>>>
>>> On 05/17/2013 07:42 AM, Emmanuel Medernach wrote:
>>>> Hello Douglas,
>>>>
>>>> We are trying to integrate the duplication in our testing and we encounter a
>>>> problem.
>>>>
>>>> We use the makeChunk script to generate the partitioning but then at load time I
>>>> get this error:
>>>>
>>>> _mysql_exceptions.IntegrityError: (1062, "Duplicate entry '12941982003370163'
>>>> for key 'PRIMARY'")
>>>>
>>>> because as you see there are 2 quite identical lines with the same primary key
>>>> in this file:
>>>>
>>>> File stripe_11/ObjectSelfOverlap_2205.csv :
>>>> ...
>>>> 12941982003370163,NULL,30.5893477843,0.0000326713,-69.3005814496,0.0000172098,0.000000000143286,17042495422951,30.5891823872,0.0000142131,-69.3005609638,0.0000127532,-0.0000000000361079,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,50984.37154362965,51111.06194962965,51068.037577962976,0,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,6,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1.22371e-29,1.64853e-30,3.88058e-29,1.73477e-29,1.87934e-29,2.85112e-31,NULL,50984.37154362965,51111.06194962965,NULL,NULL,0.3750
>>>>
>>>> 12,0.115315,0.522814,0.171576,0.771711,0.04157,33686276,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,2205,286
>>>> 12941982003370163,NULL,30.5893477843,0.0000326713,-69.3005814496,0.0000172098,0.000000000143286,17042495422951,30.5891823872,0.0000142131,-69.3005609638,0.0000127532,-0.0000000000361079,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,50984.37154362965,51111.06194962965,51068.037577962976,0,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,6,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1.22371e-29,1.64853e-30,3.88058e-29,1.73477e-29,1.87934e-29,2.85112e-31,NULL,50984.37154362965,51111.06194962965,NULL,NULL,0.3750
>>>>
>>>> 12,0.115315,0.522814,0.171576,0.771711,0.04157,33686276,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,2205,287
>>>> ...
>>>>
>>>> Could you please take a look ?
>>>>
>>>> Current code is in the tickets/2834 branch and here is the command used :
>>>> $ qserv-benchmark.py  -l -i 04
>>>>
>>>>
>>>> Thanks in advance for your help,
>>>> --

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1