Hello (again ;-)) Serge,

Please note that qserv-testdata.py should fail with underneath message.
This is normal. It means that integration tests have runned successfully but that some queries have different outputs in Qserv and MySql.
I think the new release will be cutted with that version of the integration tests.
Indeed, doing a precise and clearer report needs a consequent amount of work and will be done in a future ticket.

Have a nice w.e.

Fabrice

======================================================================
FAIL: test_case01 (lsst.qserv.tests.testdataset.TestDataSet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 28, in test_case01
    self._runTestCase(case_id)
  File "/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 24, in _runTestCase
    self.fail(msg)
AssertionError: Queries with different results between Qserv and MySQL : ['0001_fetchObjectById.txt', '1004_varObjects.txt', '1003_coneMagColorEmptyRes.txt', '1002_coneMagColor.txt', '0005_nonReplicatedTable.txt']

======================================================================
FAIL: test_case02 (lsst.qserv.tests.testdataset.TestDataSet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 32, in test_case02
    self._runTestCase(case_id)
  File "/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 24, in _runTestCase
    self.fail(msg)
AssertionError: Queries with different results between Qserv and MySQL : ['3011_selectObjectWithMagnitudes.txt', '3009_countObjectInRegionWithZFlux.txt', '0001_fetchObjectById.txt', '0004_fetchSourceById.txt', '3008_selectObjectWithColorMagnitudeGreaterThan.txt', '3012_selectObjectInCircularRegion.txt', '1004_varObjects.txt', '3001_query_035.txt', '1003_coneMagColorEmptyRes.txt', '3006_selectIntervalMagnitudes.txt', '1002_coneMagColor.txt', '3007_countObjectWithColorFluxGreaterThan.txt', '3005_objectCount.txt']

======================================================================
FAIL: test_case03 (lsst.qserv.tests.testdataset.TestDataSet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 36, in test_case03
    self._runTestCase(case_id)
  File "/opt/qserv-dev/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 24, in _runTestCase
    self.fail(msg)
AssertionError: Queries with different results between Qserv and MySQL : ['0011_selectDeepCoadd.txt', '0009_selectCCDExposure.txt', '0023_selectScienceCCDExposureWithFilternameFieldCamcolRun.txt', '0002_fetchRunAndFieldById.txt', '0030_selectScienceCCDExposureByRunField.txt', '0025_selectScienceCCDExposureWithFilternameFieldCamcolRun.txt', '0002.1_fetchRunAndFieldById.txt', '0012_selectDistinctDeepCoaddWithGivenTractPatchFiltername.txt', '0022_selectScienceCCDExposureWithFilternameFieldCamcolRun.txt', '0013_selectDeepCoadd2.txt', '0002.2_fetchRunAndFieldById.txt', '0021_selectScienceCCDExposure.txt', '0006_selectExposure.txt', '0028_selectScienceCCDExposure.txt', '0018_selectDeepCoaddWithGivenTractPatchFiltername.txt', '0024_selectDeepForcedSourceScienceCCDExposureWithFilternameFieldCamcolRun.txt', '0014_selectDeepCoadd3.txt']

----------------------------------------------------------------------



On 12/06/2013 12:19 PM, Fabrice Jammes wrote:
[log in to unmask]" type="cite">
Hello Serge,

Thanks for reporting this problem.
The problem is that qms server was using system based mysql client, due to the new version of qms init.d script.
I hadn't noticed this until i remove this system based mysql client (sudo yum remove mysql). That's why my tests when merging to master were fine.

This is fixed now.

Have a nice day,

Fabrice

On 12/05/2013 10:24 PM, Serge Monkewitz wrote:
[log in to unmask]" type="cite"> Hi Fabrice,

    I ran (per instructions at https://dev.lsstcorp.org/trac/wiki/db/Qserv/SmallDatasetBenchmarks and https://dev.lsstcorp.org/trac/wiki/db/Qserv/initd):

> qserv-start
> qserv-testdata.py

I get:

2013-12-05 15:18:11,991 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py:143} INFO Unzipping: /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/DeepCoadd_Metadata.txt.gz into /u2/smm/qserv-master-test/tmp/qservTest_case03/DeepCoadd_Metadata.txt
2013-12-05 15:18:11,992 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197} INFO cmd : gunzip -c /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/DeepCoadd_Metadata.txt.gz
2013-12-05 15:18:11,997 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py:65} INFO Creating and loading non-partitioned table DeepCoadd_Metadata
2013-12-05 15:18:11,998 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197} INFO cmd : /u2/smm/qserv-master-test/bin/mysql --sock=/u2/smm/qserv-master-test/var/lib/mysql/mysql.sock --user=root --password=pass --batch LSST -e SOURCE /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/DeepCoadd_Metadata.sql
2013-12-05 15:18:12,092 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197} INFO cmd : /u2/smm/qserv-master-test/bin/mysql --sock=/u2/smm/qserv-master-test/var/lib/mysql/mysql.sock --user=root --password=pass --batch LSST -e LOAD DATA LOCAL INFILE '/u2/smm/qserv-master-test/tmp/qservTest_case03/DeepCoadd_Metadata.txt' INTO TABLE DeepCoadd_Metadata FIELDS TERMINATED BY ',';
2013-12-05 15:18:12,109 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py:62} INFO Creating schema for table DeepSource as a view
2013-12-05 15:18:12,110 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197} INFO cmd : /u2/smm/qserv-master-test/bin/mysql --sock=/u2/smm/qserv-master-test/var/lib/mysql/mysql.sock --user=root --password=pass --batch LSST -e SOURCE /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/DeepSource.sql
2013-12-05 15:18:12,167 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py:143} INFO Unzipping: /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/Filter.txt.gz into /u2/smm/qserv-master-test/tmp/qservTest_case03/Filter.txt
2013-12-05 15:18:12,168 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197} INFO cmd : gunzip -c /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/Filter.txt.gz
2013-12-05 15:18:12,173 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py:65} INFO Creating and loading non-partitioned table Filter
2013-12-05 15:18:12,173 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197} INFO cmd : /u2/smm/qserv-master-test/bin/mysql --sock=/u2/smm/qserv-master-test/var/lib/mysql/mysql.sock --user=root --password=pass --batch LSST -e SOURCE /u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/Filter.sql
2013-12-05 15:18:12,285 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197} INFO cmd : /u2/smm/qserv-master-test/bin/mysql --sock=/u2/smm/qserv-master-test/var/lib/mysql/mysql.sock --user=root --password=pass --batch LSST -e LOAD DATA LOCAL INFILE '/u2/smm/qserv-master-test/tmp/qservTest_case03/Filter.txt' INTO TABLE Filter FIELDS TERMINATED BY ',';
2013-12-05 15:18:12,293 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:197} INFO cmd : /u2/smm/qserv-master-test/qserv/admin/bin/qms_setup.sh /u2/smm/qserv-master-test LSST case03
2013-12-05 15:18:12,590 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:239} INFO stdout : Checking if meta db LSST exists :
     /u2/smm/qserv-master-test/bin/python /u2/smm/qserv-master-test/qserv/meta/bin/metaClientTool.py --auth=/usr/local/home/smm/.lsst/qmsadm checkDbExists LSST
Already existing database LSST : yes

Dropping previous meta db LSST
     /u2/smm/qserv-master-test/bin/python /u2/smm/qserv-master-test/qserv/meta/bin/metaClientTool.py --auth=/usr/local/home/smm/.lsst/qmsadm dropDb LSST
Database dropped.

Creating meta db LSST
     /u2/smm/qserv-master-test/bin/python /u2/smm/qserv-master-test/qserv/meta/bin/metaClientTool.py --auth=/usr/local/home/smm/.lsst/qmsadm createDb LSST @/u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/db.params
Database successfully created.
Specific loading order required for meta files

Creating meta table AvgForcedPhot.params
     /u2/smm/qserv-master-test/bin/python /u2/smm/qserv-master-test/qserv/meta/bin/metaClientTool.py --auth=/usr/local/home/smm/.lsst/qmsadm createTable LSST @/u2/smm/qserv-master-test/qserv/tests/testdata/case03/data/AvgForcedPhot.params
{'partitioning': 'on', 'logicalPart': '1', 'thetaColName': 'decl', 'tableName': 'AvgForcedPhot', 'physChunking': '0x0011', 'overlap': '0.0', 'phiColName': 'ra', 'objIdColName': 'deepSourceId', 'schemaFile': 'AvgForcedPhot.sql'}
Error: Internal error. (errno:9999)

 

2013-12-05 15:18:12,591 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:241} INFO stderr : /u2/smm/qserv-master-test/qserv/admin/bin/qms_setup.sh: line 77: [: too many arguments
error (errno:63) in step : Creating meta table AvgForcedPhot.params

 

2013-12-05 15:18:12,591 {/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py:244} CRITICAL Error code returned by command : /u2/smm/qserv-master-test/qserv/admin/bin/qms_setup.sh /u2/smm/qserv-master-test LSST case03 
ERROR

======================================================================
ERROR: test_case01 (lsst.qserv.tests.testdataset.TestDataSet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 28, in test_case01
    self._runTestCase(case_id)
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 18, in _runTestCase
    bench.run(self.modeList, self.loadData)
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py", line 226, in run
    self.finalize()
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py", line 198, in finalize
    self.dataLoader[self._mode].createQmsDatabase()
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py", line 51, in createQmsDatabase
    out = commons.run_command(qms_setup_cmd)
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py", line 245, in run_command
    sys.exit(1)
SystemExit: 1

======================================================================
ERROR: test_case02 (lsst.qserv.tests.testdataset.TestDataSet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 32, in test_case02
    self._runTestCase(case_id)
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 18, in _runTestCase
    bench.run(self.modeList, self.loadData)
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py", line 226, in run
    self.finalize()
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py", line 198, in finalize
    self.dataLoader[self._mode].createQmsDatabase()
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py", line 51, in createQmsDatabase
    out = commons.run_command(qms_setup_cmd)
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py", line 245, in run_command
    sys.exit(1)
SystemExit: 1

======================================================================
ERROR: test_case03 (lsst.qserv.tests.testdataset.TestDataSet)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 36, in test_case03
    self._runTestCase(case_id)
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/testdataset.py", line 18, in _runTestCase
    bench.run(self.modeList, self.loadData)
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py", line 226, in run
    self.finalize()
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/tests/benchmark.py", line 198, in finalize
    self.dataLoader[self._mode].createQmsDatabase()
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/qservdataloader.py", line 51, in createQmsDatabase
    out = commons.run_command(qms_setup_cmd)
  File "/u2/smm/qserv-master-test/lib/python2.6/site-packages/lsst/qserv/admin/commons.py", line 245, in run_command
    sys.exit(1)
SystemExit: 1

----------------------------------------------------------------------
Ran 3 tests in 123.544s


Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1



Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1



Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1