Hello (again ;-)) Serge,
Please note that qserv-testdata.py should fail with underneath message.
This is normal. It means that integration tests have runned successfully
but that some queries have different outputs in Qserv and MySql.
I think the new release will be cutted with that version of the
integration tests.
Indeed, doing a precise and clearer report needs a consequent amount of
work and will be done in a future ticket.
Have a nice w.e.
Fabrice
======================================================================
FAIL: test_case01 (lsst.qserv.tests.testdataset.TestDataSet)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
line 28, in test_case01
self._runTestCase(case_id)
File
"/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
line 24, in _runTestCase
self.fail(msg)
AssertionError: Queries with different results between Qserv and MySQL :
['0001_fetchObjectById.txt', '1004_varObjects.txt',
'1003_coneMagColorEmptyRes.txt', '1002_coneMagColor.txt',
'0005_nonReplicatedTable.txt']
======================================================================
FAIL: test_case02 (lsst.qserv.tests.testdataset.TestDataSet)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
line 32, in test_case02
self._runTestCase(case_id)
File
"/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
line 24, in _runTestCase
self.fail(msg)
AssertionError: Queries with different results between Qserv and MySQL :
['3011_selectObjectWithMagnitudes.txt',
'3009_countObjectInRegionWithZFlux.txt', '0001_fetchObjectById.txt',
'0004_fetchSourceById.txt',
'3008_selectObjectWithColorMagnitudeGreaterThan.txt',
'3012_selectObjectInCircularRegion.txt', '1004_varObjects.txt',
'3001_query_035.txt', '1003_coneMagColorEmptyRes.txt',
'3006_selectIntervalMagnitudes.txt', '1002_coneMagColor.txt',
'3007_countObjectWithColorFluxGreaterThan.txt', '3005_objectCount.txt']
======================================================================
FAIL: test_case03 (lsst.qserv.tests.testdataset.TestDataSet)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
line 36, in test_case03
self._runTestCase(case_id)
File
"/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
line 24, in _runTestCase
self.fail(msg)
AssertionError: Queries with different results between Qserv and MySQL :
['0011_selectDeepCoadd.txt', '0009_selectCCDExposure.txt',
'0023_selectScienceCCDExposureWithFilternameFieldCamcolRun.txt',
'0002_fetchRunAndFieldById.txt',
'0030_selectScienceCCDExposureByRunField.txt',
'0025_selectScienceCCDExposureWithFilternameFieldCamcolRun.txt',
'0002.1_fetchRunAndFieldById.txt',
'0012_selectDistinctDeepCoaddWithGivenTractPatchFiltername.txt',
'0022_selectScienceCCDExposureWithFilternameFieldCamcolRun.txt',
'0013_selectDeepCoadd2.txt', '0002.2_fetchRunAndFieldById.txt',
'0021_selectScienceCCDExposure.txt', '0006_selectExposure.txt',
'0028_selectScienceCCDExposure.txt',
'0018_selectDeepCoaddWithGivenTractPatchFiltername.txt',
'0024_selectDeepForcedSourceScienceCCDExposureWithFilternameFieldCamcolRun.txt',
'0014_selectDeepCoadd3.txt']
----------------------------------------------------------------------
On 12/06/2013 12:19 PM, Fabrice Jammes wrote:
> Hello Serge,
>
> Thanks for reporting this problem.
> The problem is that qms server was using system based mysql client,
> due to the new version of qms init.d script.
> I hadn't noticed this until i remove this system based mysql client
> (sudo yum remove mysql). That's why my tests when merging to master
> were fine.
>
> This is fixed now.
>
> Have a nice day,
>
> Fabrice
>
> On 12/05/2013 10:24 PM, Serge Monkewitz wrote:
>> Hi Fabrice,
>>
>> I ran (per instructions at
>> https://dev.lsstcorp.org/trac/wiki/db/Qserv/SmallDatasetBenchmarks and https://dev.lsstcorp.org/trac/wiki/db/Qserv/initd):
>>
>> > qserv-start
>> > qserv-testdata.py
>>
>> I get:
>>
>> 2013-12-05 15:18:11,991
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py:143}
>> INFO Unzipping:
>> /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/DeepCoadd_Metadata.txt.gz
>> into
>> /u2/smm/qserv-master-test/tmp/qservTest_case03/DeepCoadd_Metadata.txt
>> 2013-12-05 15:18:11,992
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197}
>> INFO cmd : gunzip -c
>> /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/DeepCoadd_Metadata.txt.gz
>> 2013-12-05 15:18:11,997
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py:65}
>> INFO Creating and loading non-partitioned table DeepCoadd_Metadata
>> 2013-12-05 15:18:11,998
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197}
>> INFO cmd : /u2/smm/qserv-master-test/bin/mysql
>> --sock=/u2/smm/qserv-master-test/var/lib/mysql/mysql.sock --user=root
>> --password=pass --batch LSST -e SOURCE
>> /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/DeepCoadd_Metadata.sql
>> 2013-12-05 15:18:12,092
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197}
>> INFO cmd : /u2/smm/qserv-master-test/bin/mysql
>> --sock=/u2/smm/qserv-master-test/var/lib/mysql/mysql.sock --user=root
>> --password=pass --batch LSST -e LOAD DATA LOCAL INFILE
>> '/u2/smm/qserv-master-test/tmp/qservTest_case03/DeepCoadd_Metadata.txt'
>> INTO TABLE DeepCoadd_Metadata FIELDS TERMINATED BY ',';
>> 2013-12-05 15:18:12,109
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py:62}
>> INFO Creating schema for table DeepSource as a view
>> 2013-12-05 15:18:12,110
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197}
>> INFO cmd : /u2/smm/qserv-master-test/bin/mysql
>> --sock=/u2/smm/qserv-master-test/var/lib/mysql/mysql.sock --user=root
>> --password=pass --batch LSST -e SOURCE
>> /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/DeepSource.sql
>> 2013-12-05 15:18:12,167
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py:143}
>> INFO Unzipping:
>> /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/Filter.txt.gz
>> into /u2/smm/qserv-master-test/tmp/qservTest_case03/Filter.txt
>> 2013-12-05 15:18:12,168
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197}
>> INFO cmd : gunzip -c
>> /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/Filter.txt.gz
>> 2013-12-05 15:18:12,173
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py:65}
>> INFO Creating and loading non-partitioned table Filter
>> 2013-12-05 15:18:12,173
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197}
>> INFO cmd : /u2/smm/qserv-master-test/bin/mysql
>> --sock=/u2/smm/qserv-master-test/var/lib/mysql/mysql.sock --user=root
>> --password=pass --batch LSST -e SOURCE
>> /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/Filter.sql
>> 2013-12-05 15:18:12,285
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197}
>> INFO cmd : /u2/smm/qserv-master-test/bin/mysql
>> --sock=/u2/smm/qserv-master-test/var/lib/mysql/mysql.sock --user=root
>> --password=pass --batch LSST -e LOAD DATA LOCAL INFILE
>> '/u2/smm/qserv-master-test/tmp/qservTest_case03/Filter.txt' INTO
>> TABLE Filter FIELDS TERMINATED BY ',';
>> 2013-12-05 15:18:12,293
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197}
>> INFO cmd : /u2/smm/qserv-master-test/qserv/admin/bin/qms_setup.sh
>> /u2/smm/qserv-master-test LSST case03
>> 2013-12-05 15:18:12,590
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:239}
>> INFO stdout : Checking if meta db LSST exists :
>> /u2/smm/qserv-master-test/bin/python
>> /u2/smm/qserv-master-test/qserv/meta/bin/metaClientTool.py
>> --auth=/usr/local/home/smm/.lsst/qmsadm checkDbExists LSST
>> Already existing database LSST : yes
>>
>> Dropping previous meta db LSST
>> /u2/smm/qserv-master-test/bin/python
>> /u2/smm/qserv-master-test/qserv/meta/bin/metaClientTool.py
>> --auth=/usr/local/home/smm/.lsst/qmsadm dropDb LSST
>> Database dropped.
>>
>> Creating meta db LSST
>> /u2/smm/qserv-master-test/bin/python
>> /u2/smm/qserv-master-test/qserv/meta/bin/metaClientTool.py
>> --auth=/usr/local/home/smm/.lsst/qmsadm createDb LSST
>> @/u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/db.params
>> Database successfully created.
>> Specific loading order required for meta files
>>
>> Creating meta table AvgForcedPhot.params
>> /u2/smm/qserv-master-test/bin/python
>> /u2/smm/qserv-master-test/qserv/meta/bin/metaClientTool.py
>> --auth=/usr/local/home/smm/.lsst/qmsadm createTable LSST
>> @/u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/AvgForcedPhot.params
>> {'partitioning': 'on', 'logicalPart': '1', 'thetaColName': 'decl',
>> 'tableName': 'AvgForcedPhot', 'physChunking': '0x0011', 'overlap':
>> '0.0', 'phiColName': 'ra', 'objIdColName': 'deepSourceId',
>> 'schemaFile': 'AvgForcedPhot.sql'}
>> Error: Internal error. (errno:9999)
>>
>>
>> 2013-12-05 15:18:12,591
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:241}
>> INFO stderr : /u2/smm/qserv-master-test/qserv/admin/bin/qms_setup.sh:
>> line 77: [: too many arguments
>> error (errno:63) in step : Creating meta table AvgForcedPhot.params
>>
>>
>> 2013-12-05 15:18:12,591
>> {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:244}
>> CRITICAL Error code returned by command :
>> /u2/smm/qserv-master-test/qserv/admin/bin/qms_setup.sh
>> /u2/smm/qserv-master-test LSST case03
>> ERROR
>>
>> ======================================================================
>> ERROR: test_case01 (lsst.qserv.tests.testdataset.TestDataSet)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
>> line 28, in test_case01
>> self._runTestCase(case_id)
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
>> line 18, in _runTestCase
>> bench.run(self.modeList, self.loadData)
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py",
>> line 226, in run
>> self.finalize()
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py",
>> line 198, in finalize
>> self.dataLoader[self._mode].createQmsDatabase()
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py",
>> line 51, in createQmsDatabase
>> out = commons.run_command(qms_setup_cmd)
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py",
>> line 245, in run_command
>> sys.exit(1)
>> SystemExit: 1
>>
>> ======================================================================
>> ERROR: test_case02 (lsst.qserv.tests.testdataset.TestDataSet)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
>> line 32, in test_case02
>> self._runTestCase(case_id)
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
>> line 18, in _runTestCase
>> bench.run(self.modeList, self.loadData)
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py",
>> line 226, in run
>> self.finalize()
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py",
>> line 198, in finalize
>> self.dataLoader[self._mode].createQmsDatabase()
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py",
>> line 51, in createQmsDatabase
>> out = commons.run_command(qms_setup_cmd)
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py",
>> line 245, in run_command
>> sys.exit(1)
>> SystemExit: 1
>>
>> ======================================================================
>> ERROR: test_case03 (lsst.qserv.tests.testdataset.TestDataSet)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
>> line 36, in test_case03
>> self._runTestCase(case_id)
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py",
>> line 18, in _runTestCase
>> bench.run(self.modeList, self.loadData)
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py",
>> line 226, in run
>> self.finalize()
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py",
>> line 198, in finalize
>> self.dataLoader[self._mode].createQmsDatabase()
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py",
>> line 51, in createQmsDatabase
>> out = commons.run_command(qms_setup_cmd)
>> File
>> "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py",
>> line 245, in run_command
>> sys.exit(1)
>> SystemExit: 1
>>
>> ----------------------------------------------------------------------
>> Ran 3 tests in 123.544s
>>
>>
>> ------------------------------------------------------------------------
>>
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the QSERV-L list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>>
>
>
> ------------------------------------------------------------------------
>
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the QSERV-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
