 |
 |
The other thing to remember is that objectId is not a primary key in Object_*Overlap* tables. Recall that the concept of an overlap allows "duplicate" rows that differ only in subChunkId. This allows simpler/faster on-the-fly creation of subchunk tables. We haven't thought about how this pseudo-duplication affects schema handling yet (because we have put very little thought into schema handling for non baseline LSST schema). -Daniel On 05/17/2013 10:30 AM, Douglas Smith wrote: > I thought we had that fixed in the latest versions? But we decided in > the past that for duplicated data for testing we aren't concerned about > this. That we just need to serve a large amount of data. And you can > just add "IGNORE" to the insert statement. > > Oh, and also, this is the overlap tables. Yes, on the overlap tables, this > will be true, and you need to remove the primary key from the table > for the overlap tables. And we only need the overlap tables for the > Object table, not for the source table. > > Douglas > > > On 05/17/2013 07:42 AM, Emmanuel Medernach wrote: >> Hello Douglas, >> >> We are trying to integrate the duplication in our testing and we encounter a >> problem. >> >> We use the makeChunk script to generate the partitioning but then at load time I >> get this error: >> >> _mysql_exceptions.IntegrityError: (1062, "Duplicate entry '12941982003370163' >> for key 'PRIMARY'") >> >> because as you see there are 2 quite identical lines with the same primary key >> in this file: >> >> File stripe_11/ObjectSelfOverlap_2205.csv : >> ... >> 12941982003370163,NULL,30.5893477843,0.0000326713,-69.3005814496,0.0000172098,0.000000000143286,17042495422951,30.5891823872,0.0000142131,-69.3005609638,0.0000127532,-0.0000000000361079,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,50984.37154362965,51111.06194962965,51068.037577962976,0,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,6,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1.22371e-29,1.64853e-30,3.88058e-29,1.73477e-29,1.87934e-29,2.85112e-31,NULL,50984.37154362965,51111.06194962965,NULL,NULL,0.3750 >> >> 12,0.115315,0.522814,0.171576,0.771711,0.04157,33686276,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,2205,286 >> 12941982003370163,NULL,30.5893477843,0.0000326713,-69.3005814496,0.0000172098,0.000000000143286,17042495422951,30.5891823872,0.0000142131,-69.3005609638,0.0000127532,-0.0000000000361079,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,50984.37154362965,51111.06194962965,51068.037577962976,0,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,6,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,1.22371e-29,1.64853e-30,3.88058e-29,1.73477e-29,1.87934e-29,2.85112e-31,NULL,50984.37154362965,51111.06194962965,NULL,NULL,0.3750 >> >> 12,0.115315,0.522814,0.171576,0.771711,0.04157,33686276,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,2205,287 >> ... >> >> Could you please take a look ? >> >> Current code is in the tickets/2834 branch and here is the command used : >> $ qserv-benchmark.py -l -i 04 >> >> >> Thanks in advance for your help, >> -- > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the QSERV-L list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the QSERV-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1