LISTSERV mailing list manager LISTSERV 16.5

Help for QSERV-L Archives


QSERV-L Archives

QSERV-L Archives


QSERV-L@LISTSERV.SLAC.STANFORD.EDU


View:

Message:

[

First

|

Previous

|

Next

|

Last

]

By Topic:

[

First

|

Previous

|

Next

|

Last

]

By Author:

[

First

|

Previous

|

Next

|

Last

]

Font:

Proportional Font

LISTSERV Archives

LISTSERV Archives

QSERV-L Home

QSERV-L Home

QSERV-L  January 2015

QSERV-L January 2015

Subject:

Composite index better than memory tables?

From:

Fabrice Jammes <[log in to unmask]>

Reply-To:

General discussion for qserv (LSST prototype baseline catalog)

Date:

Thu, 8 Jan 2015 17:46:21 -0800

Content-Type:

multipart/mixed

Parts/Attachments:

Parts/Attachments

text/plain (76 lines) , compIdx.py (106 lines)

Hi all,

Here's a small report which show that it's worth comparing composite 
index with memory tables.
It would be worth to experiment on wider tables with more column.

Program is attached (I hope it is correct...), code is below.

Cheers,


create table X1 (objectId BIGINT, chunkId INT, subChunkId INT, 
index(objectId), index(subChunkId)) ENGINE=MyISAM
create table X2 (objectId BIGINT, chunkId INT, subChunkId INT, 
index(objectId, subChunkId)) ENGINE=MyISAM
create table X3 (objectId BIGINT, chunkId INT, subChunkId INT, 
index(subChunkId, objectId)) ENGINE=MyISAM
Lines per table 90000
Launching ['select * from X1 where subChunkId = %s'], 300 times against X1
Elapsed time 0.456371068954

Launching ['select * from X2 where subChunkId = %s'], 300 times against X2
Elapsed time 1.83970189095

Launching ['select * from X3 where subChunkId = %s'], 300 times against X3
Elapsed time 0.513654947281

Launching ['select * from X1 where subChunkId = %s'], 300 times against X1
Elapsed time 0.514945983887

Launching ['select * from X2 where subChunkId = %s'], 300 times against X2
Elapsed time 1.85265922546

Launching ['select * from X3 where subChunkId = %s'], 300 times against X3
Elapsed time 0.534698009491

Launching ['select o1.objectId, o2.objectId from X1 o1, X1 o2 where 
o1.subChunkId = %s and o2.subChunkId = %s and o1.objectId <> 
o2.objectId'], 10 times against X1
Elapsed time 1.41969299316

Launching ['select o1.objectId, o2.objectId from X2 o1, X2 o2 where 
o1.subChunkId = %s and o2.subChunkId = %s and o1.objectId <> 
o2.objectId'], 10 times against X2
Elapsed time 1.15225410461

Launching ['select o1.objectId, o2.objectId from X3 o1, X3 o2 where 
o1.subChunkId = %s and o2.subChunkId = %s and o1.objectId <> 
o2.objectId'], 10 times against X3
Elapsed time 1.15079092979

Launching ['CREATE TABLE dummy SELECT * FROM X1 WHERE subChunkId = %s', 
'select o1.objectId, o2.objectId from dummy o1, dummy o2 where 
o1.subChunkId = %s and o2.subChunkId = %s and o1.objectId <> 
o2.objectId', 'DROP TABLE dummy'], 10 times against X1
Elapsed time 1.43273210526

Launching ['CREATE TABLE dummy SELECT * FROM X2 WHERE subChunkId = %s', 
'select o1.objectId, o2.objectId from dummy o1, dummy o2 where 
o1.subChunkId = %s and o2.subChunkId = %s and o1.objectId <> 
o2.objectId', 'DROP TABLE dummy'], 10 times against X2
Elapsed time 1.356112957

Launching ['CREATE TABLE dummy SELECT * FROM X3 WHERE subChunkId = %s', 
'select o1.objectId, o2.objectId from dummy o1, dummy o2 where 
o1.subChunkId = %s and o2.subChunkId = %s and o1.objectId <> 
o2.objectId', 'DROP TABLE dummy'], 10 times against X3
Elapsed time 1.33042907715

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1

Top of Message | Previous Page | Permalink

Advanced Options


Options

Log In

Log In

Get Password

Get Password


Search Archives

Search Archives


Subscribe or Unsubscribe

Subscribe or Unsubscribe


Archives

March 2018
February 2018
January 2018
December 2017
August 2017
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
December 2012

ATOM RSS1 RSS2



LISTSERV.SLAC.STANFORD.EDU

Secured by F-Secure Anti-Virus CataList Email List Search Powered by the LISTSERV Email List Manager

Privacy Notice, Security Notice and Terms of Use