John, can you resent it as an attachment in a file, I'm afraid something may be lost if I copy and paste it from Outlook.
Cheers,
Andy
From: Gates, John H
Sent: Tuesday, November 10, 2015 2:39 PM
To: Salnikov, Andrei A.; Fabrice Jammes; qserv-l
Subject: Re: [QSERV-L] Upgrade of IN2P3 css metadata
Hi Andy,
Here's the output:
qserv > DUMP EVERYTHING
~ ;
/
/DBS
/DBS/LSST.LOCK
/DBS/LSST.json {"releaseStatus": "UNRELEASED", "partitioningId": "0000000000", "uuid": "e0f64473-b8b4-4ead-ac8f-092c1feea554", "storageClass": "L2"}
/DBS/LSST READY
/DBS/LSST/TABLES
/DBS/LSST/TABLES/Source READY
/DBS/LSST/TABLES/Source/partitioning.json {"dirTable": "Object", "overlap": "0.0", "latColName": "coord_decl", "lonColName": "coord_ra", "subChunks": "0", "dirColName": "objectId"}
/DBS/LSST/TABLES/Object.json {"compression": "0", "match": "0", "schema": "(\n `deepSourceId` bigint(20) NOT NULL,\n `ra` double NOT NULL,\n `decl` double NOT NULL,\n `raVar` double DEFAULT NULL,\n `declVar` double DEFAULT NULL,\n `radeclCov` double
DEFAULT NULL,\n `chunkId` int(11) NOT NULL,\n `subChunkId` int(11) NOT NULL,\n `u_psfFlux` double DEFAULT NULL,\n `u_psfFluxSigma` double DEFAULT NULL,\n `u_apFlux` double DEFAULT NULL,\n `u_apFluxSigma` double DEFAULT NULL,\n `u_modelFlux` double DEFAULT
NULL,\n `u_modelFluxSigma` double DEFAULT NULL,\n `u_instFlux` double DEFAULT NULL,\n `u_instFluxSigma` double DEFAULT NULL,\n `u_apCorrection` double DEFAULT NULL,\n `u_apCorrectionSigma` double DEFAULT NULL,\n `u_shapeIx` double DEFAULT NULL,\n `u_shapeIy`
double DEFAULT NULL,\n `u_shapeIxVar` double DEFAULT NULL,\n `u_shapeIyVar` double DEFAULT NULL,\n `u_shapeIxIyCov` double DEFAULT NULL,\n `u_shapeIxx` double DEFAULT NULL,\n `u_shapeIyy` double DEFAULT NULL,\n `u_shapeIxy` double DEFAULT NULL,\n `u_shapeIxxVar`
double DEFAULT NULL,\n `u_shapeIyyVar` double DEFAULT NULL,\n `u_shapeIxyVar` double DEFAULT NULL,\n `u_shapeIxxIyyCov` double DEFAULT NULL,\n `u_shapeIxxIxyCov` double DEFAULT NULL,\n `u_shapeIyyIxyCov` double DEFAULT NULL,\n `u_extendedness` double
DEFAULT NULL,\n `u_flagNegative` bit(1) DEFAULT NULL,\n `u_flagBadMeasCentroid` bit(1) DEFAULT NULL,\n `u_flagPixEdge` bit(1) DEFAULT NULL,\n `u_flagPixInterpAny` bit(1) DEFAULT NULL,\n `u_flagPixInterpCen` bit(1) DEFAULT NULL,\n `u_flagPixSaturAny`
bit(1) DEFAULT NULL,\n `u_flagPixSaturCen` bit(1) DEFAULT NULL,\n `u_flagBadPsfFlux` bit(1) DEFAULT NULL,\n `u_flagBadApFlux` bit(1) DEFAULT NULL,\n `u_flagBadModelFlux` bit(1) DEFAULT NULL,\n `u_flagBadInstFlux` bit(1) DEFAULT NULL,\n `u_flagBadCentroid`
bit(1) DEFAULT NULL,\n `u_flagBadShape` bit(1) DEFAULT NULL,\n `g_psfFlux` double DEFAULT NULL,\n `g_psfFluxSigma` double DEFAULT NULL,\n `g_apFlux` double DEFAULT NULL,\n `g_apFluxSigma` double DEFAULT NULL,\n `g_modelFlux` double DEFAULT NULL,\n `g_modelFluxSigma`
double DEFAULT NULL,\n `g_instFlux` double DEFAULT NULL,\n `g_instFluxSigma` double DEFAULT NULL,\n `g_apCorrection` double DEFAULT NULL,\n `g_apCorrectionSigma` double DEFAULT NULL,\n `g_shapeIx` double DEFAULT NULL,\n `g_shapeIy` double DEFAULT NULL,\n
`g_shapeIxVar` double DEFAULT NULL,\n `g_shapeIyVar` double DEFAULT NULL,\n `g_shapeIxIyCov` double DEFAULT NULL,\n `g_shapeIxx` double DEFAULT NULL,\n `g_shapeIyy` double DEFAULT NULL,\n `g_shapeIxy` double DEFAULT NULL,\n `g_shapeIxxVar` double DEFAULT
NULL,\n `g_shapeIyyVar` double DEFAULT NULL,\n `g_shapeIxyVar` double DEFAULT NULL,\n `g_shapeIxxIyyCov` double DEFAULT NULL,\n `g_shapeIxxIxyCov` double DEFAULT NULL,\n `g_shapeIyyIxyCov` double DEFAULT NULL,\n `g_extendedness` double DEFAULT NULL,\n
`g_flagNegative` bit(1) DEFAULT NULL,\n `g_flagBadMeasCentroid` bit(1) DEFAULT NULL,\n `g_flagPixEdge` bit(1) DEFAULT NULL,\n `g_flagPixInterpAny` bit(1) DEFAULT NULL,\n `g_flagPixInterpCen` bit(1) DEFAULT NULL,\n `g_flagPixSaturAny` bit(1) DEFAULT NULL,\n
`g_flagPixSaturCen` bit(1) DEFAULT NULL,\n `g_flagBadPsfFlux` bit(1) DEFAULT NULL,\n `g_flagBadApFlux` bit(1) DEFAULT NULL,\n `g_flagBadModelFlux` bit(1) DEFAULT NULL,\n `g_flagBadInstFlux` bit(1) DEFAULT NULL,\n `g_flagBadCentroid` bit(1) DEFAULT NULL,\n
`g_flagBadShape` bit(1) DEFAULT NULL,\n `r_psfFlux` double DEFAULT NULL,\n `r_psfFluxSigma` double DEFAULT NULL,\n `r_apFlux` double DEFAULT NULL,\n `r_apFluxSigma` double DEFAULT NULL,\n `r_modelFlux` double DEFAULT NULL,\n `r_modelFluxSigma` double
DEFAULT NULL,\n `r_instFlux` double DEFAULT NULL,\n `r_instFluxSigma` double DEFAULT NULL,\n `r_apCorrection` double DEFAULT NULL,\n `r_apCorrectionSigma` double DEFAULT NULL,\n `r_shapeIx` double DEFAULT NULL,\n `r_shapeIy` double DEFAULT NULL,\n `r_shapeIxVar`
double DEFAULT NULL,\n `r_shapeIyVar` double DEFAULT NULL,\n `r_shapeIxIyCov` double DEFAULT NULL,\n `r_shapeIxx` double DEFAULT NULL,\n `r_shapeIyy` double DEFAULT NULL,\n `r_shapeIxy` double DEFAULT NULL,\n `r_shapeIxxVar` double DEFAULT NULL,\n `r_shapeIyyVar`
double DEFAULT NULL,\n `r_shapeIxyVar` double DEFAULT NULL,\n `r_shapeIxxIyyCov` double DEFAULT NULL,\n `r_shapeIxxIxyCov` double DEFAULT NULL,\n `r_shapeIyyIxyCov` double DEFAULT NULL,\n `r_extendedness` double DEFAULT NULL,\n `r_flagNegative` bit(1)
DEFAULT NULL,\n `r_flagBadMeasCentroid` bit(1) DEFAULT NULL,\n `r_flagPixEdge` bit(1) DEFAULT NULL,\n `r_flagPixInterpAny` bit(1) DEFAULT NULL,\n `r_flagPixInterpCen` bit(1) DEFAULT NULL,\n `r_flagPixSaturAny` bit(1) DEFAULT NULL,\n `r_flagPixSaturCen`
bit(1) DEFAULT NULL,\n `r_flagBadPsfFlux` bit(1) DEFAULT NULL,\n `r_flagBadApFlux` bit(1) DEFAULT NULL,\n `r_flagBadModelFlux` bit(1) DEFAULT NULL,\n `r_flagBadInstFlux` bit(1) DEFAULT NULL,\n `r_flagBadCentroid` bit(1) DEFAULT NULL,\n `r_flagBadShape`
bit(1) DEFAULT NULL,\n `i_psfFlux` double DEFAULT NULL,\n `i_psfFluxSigma` double DEFAULT NULL,\n `i_apFlux` double DEFAULT NULL,\n `i_apFluxSigma` double DEFAULT NULL,\n `i_modelFlux` double DEFAULT NULL,\n `i_modelFluxSigma` double DEFAULT NULL,\n
`i_instFlux` double DEFAULT NULL,\n `i_instFluxSigma` double DEFAULT NULL,\n `i_apCorrection` double DEFAULT NULL,\n `i_apCorrectionSigma` double DEFAULT NULL,\n `i_shapeIx` double DEFAULT NULL,\n `i_shapeIy` double DEFAULT NULL,\n `i_shapeIxVar` double
DEFAULT NULL,\n `i_shapeIyVar` double DEFAULT NULL,\n `i_shapeIxIyCov` double DEFAULT NULL,\n `i_shapeIxx` double DEFAULT NULL,\n `i_shapeIyy` double DEFAULT NULL,\n `i_shapeIxy` double DEFAULT NULL,\n `i_shapeIxxVar` double DEFAULT NULL,\n `i_shapeIyyVar`
double DEFAULT NULL,\n `i_shapeIxyVar` double DEFAULT NULL,\n `i_shapeIxxIyyCov` double DEFAULT NULL,\n `i_shapeIxxIxyCov` double DEFAULT NULL,\n `i_shapeIyyIxyCov` double DEFAULT NULL,\n `i_extendedness` double DEFAULT NULL,\n `i_flagNegative` bit(1)
DEFAULT NULL,\n `i_flagBadMeasCentroid` bit(1) DEFAULT NULL,\n `i_flagPixEdge` bit(1) DEFAULT NULL,\n `i_flagPixInterpAny` bit(1) DEFAULT NULL,\n `i_flagPixInterpCen` bit(1) DEFAULT NULL,\n `i_flagPixSaturAny` bit(1) DEFAULT NULL,\n `i_flagPixSaturCen`
bit(1) DEFAULT NULL,\n `i_flagBadPsfFlux` bit(1) DEFAULT NULL,\n `i_flagBadApFlux` bit(1) DEFAULT NULL,\n `i_flagBadModelFlux` bit(1) DEFAULT NULL,\n `i_flagBadInstFlux` bit(1) DEFAULT NULL,\n `i_flagBadCentroid` bit(1) DEFAULT NULL,\n `i_flagBadShape`
bit(1) DEFAULT NULL,\n `z_psfFlux` double DEFAULT NULL,\n `z_psfFluxSigma` double DEFAULT NULL,\n `z_apFlux` double DEFAULT NULL,\n `z_apFluxSigma` double DEFAULT NULL,\n `z_modelFlux` double DEFAULT NULL,\n `z_modelFluxSigma` double DEFAULT NULL,\n
`z_instFlux` double DEFAULT NULL,\n `z_instFluxSigma` double DEFAULT NULL,\n `z_apCorrection` double DEFAULT NULL,\n `z_apCorrectionSigma` double DEFAULT NULL,\n `z_shapeIx` double DEFAULT NULL,\n `z_shapeIy` double DEFAULT NULL,\n `z_shapeIxVar` double
DEFAULT NULL,\n `z_shapeIyVar` double DEFAULT NULL,\n `z_shapeIxIyCov` double DEFAULT NULL,\n `z_shapeIxx` double DEFAULT NULL,\n `z_shapeIyy` double DEFAULT NULL,\n `z_shapeIxy` double DEFAULT NULL,\n `z_shapeIxxVar` double DEFAULT NULL,\n `z_shapeIyyVar`
double DEFAULT NULL,\n `z_shapeIxyVar` double DEFAULT NULL,\n `z_shapeIxxIyyCov` double DEFAULT NULL,\n `z_shapeIxxIxyCov` double DEFAULT NULL,\n `z_shapeIyyIxyCov` double DEFAULT NULL,\n `z_extendedness` double DEFAULT NULL,\n `z_flagNegative` bit(1)
DEFAULT NULL,\n `z_flagBadMeasCentroid` bit(1) DEFAULT NULL,\n `z_flagPixEdge` bit(1) DEFAULT NULL,\n `z_flagPixInterpAny` bit(1) DEFAULT NULL,\n `z_flagPixInterpCen` bit(1) DEFAULT NULL,\n `z_flagPixSaturAny` bit(1) DEFAULT NULL,\n `z_flagPixSaturCen`
bit(1) DEFAULT NULL,\n `z_flagBadPsfFlux` bit(1) DEFAULT NULL,\n `z_flagBadApFlux` bit(1) DEFAULT NULL,\n `z_flagBadModelFlux` bit(1) DEFAULT NULL,\n `z_flagBadInstFlux` bit(1) DEFAULT NULL,\n `z_flagBadCentroid` bit(1) DEFAULT NULL,\n `z_flagBadShape`
bit(1) DEFAULT NULL,\n `y_psfFlux` double DEFAULT NULL,\n `y_psfFluxSigma` double DEFAULT NULL,\n `y_apFlux` double DEFAULT NULL,\n `y_apFluxSigma` double DEFAULT NULL,\n `y_modelFlux` double DEFAULT NULL,\n `y_modelFluxSigma` double DEFAULT NULL,\n
`y_instFlux` double DEFAULT NULL,\n `y_instFluxSigma` double DEFAULT NULL,\n `y_apCorrection` double DEFAULT NULL,\n `y_apCorrectionSigma` double DEFAULT NULL,\n `y_shapeIx` double DEFAULT NULL,\n `y_shapeIy` double DEFAULT NULL,\n `y_shapeIxVar` double
DEFAULT NULL,\n `y_shapeIyVar` double DEFAULT NULL,\n `y_shapeIxIyCov` double DEFAULT NULL,\n `y_shapeIxx` double DEFAULT NULL,\n `y_shapeIyy` double DEFAULT NULL,\n `y_shapeIxy` double DEFAULT NULL,\n `y_shapeIxxVar` double DEFAULT NULL,\n `y_shapeIyyVar`
double DEFAULT NULL,\n `y_shapeIxyVar` double DEFAULT NULL,\n `y_shapeIxxIyyCov` double DEFAULT NULL,\n `y_shapeIxxIxyCov` double DEFAULT NULL,\n `y_shapeIyyIxyCov` double DEFAULT NULL,\n `y_extendedness` double DEFAULT NULL,\n `y_flagNegative` bit(1)
DEFAULT NULL,\n `y_flagBadMeasCentroid` bit(1) DEFAULT NULL,\n `y_flagPixEdge` bit(1) DEFAULT NULL,\n `y_flagPixInterpAny` bit(1) DEFAULT NULL,\n `y_flagPixInterpCen` bit(1) DEFAULT NULL,\n `y_flagPixSaturAny` bit(1) DEFAULT NULL,\n `y_flagPixSaturCen`
bit(1) DEFAULT NULL,\n `y_flagBadPsfFlux` bit(1) DEFAULT NULL,\n `y_flagBadApFlux` bit(1) DEFAULT NULL,\n `y_flagBadModelFlux` bit(1) DEFAULT NULL,\n `y_flagBadInstFlux` bit(1) DEFAULT NULL,\n `y_flagBadCentroid` bit(1) DEFAULT NULL,\n `y_flagBadShape`
bit(1) DEFAULT NULL,\n PRIMARY KEY (`deepSourceId`),\n KEY `subChunkId` (`subChunkId`)\n)"}
/DBS/LSST/TABLES/ForcedSource.json {"compression": "0", "match": "0", "schema": "(\n `deepSourceId` bigint(20) NOT NULL,\n `scienceCcdExposureId` bigint(20) NOT NULL,\n `psfFlux` float DEFAULT NULL,\n `psfFluxSigma` float DEFAULT NULL,\n `flagBadMeasCentroid`
bit(1) NOT NULL,\n `flagPixEdge` bit(1) NOT NULL,\n `flagPixInterpAny` bit(1) NOT NULL,\n `flagPixInterpCen` bit(1) NOT NULL,\n `flagPixSaturAny` bit(1) NOT NULL,\n `flagPixSaturCen` bit(1) NOT NULL,\n `flagBadPsfFlux` bit(1) NOT NULL,\n `chunkId` int(11)
NOT NULL,\n `subChunkId` int(11) NOT NULL,\n PRIMARY KEY (`deepSourceId`,`scienceCcdExposureId`)\n)"}
/DBS/LSST/TABLES/Object READY
/DBS/LSST/TABLES/Object/partitioning.json {"dirTable": "Object", "overlap": "0.01667", "latColName": "decl", "lonColName": "ra", "subChunks": "1", "dirColName": "deepSourceId"}
/DBS/LSST/TABLES/Source.json {"compression": "0", "match": "0", "schema": "(\n `id` bigint(20) NOT NULL,\n `coord_ra` double DEFAULT NULL,\n `coord_decl` double DEFAULT NULL,\n `coord_htmId20` bigint(20) DEFAULT NULL,\n `parent` bigint(20) DEFAULT NULL,\n
`flags_badcentroid` bit(1) NOT NULL,\n `centroid_sdss_x` double DEFAULT NULL,\n `centroid_sdss_y` double DEFAULT NULL,\n `centroid_sdss_xVar` double DEFAULT NULL,\n `centroid_sdss_xyCov` double DEFAULT NULL,\n `centroid_sdss_yVar` double DEFAULT NULL,\n
`centroid_sdss_flags` bit(1) NOT NULL,\n `flags_pixel_edge` bit(1) NOT NULL,\n `flags_pixel_interpolated_any` bit(1) NOT NULL,\n `flags_pixel_interpolated_center` bit(1) NOT NULL,\n `flags_pixel_saturated_any` bit(1) NOT NULL,\n `flags_pixel_saturated_center`
bit(1) NOT NULL,\n `flags_pixel_cr_any` bit(1) NOT NULL,\n `flags_pixel_cr_center` bit(1) NOT NULL,\n `centroid_naive_x` double DEFAULT NULL,\n `centroid_naive_y` double DEFAULT NULL,\n `centroid_naive_xVar` double DEFAULT NULL,\n `centroid_naive_xyCov`
double DEFAULT NULL,\n `centroid_naive_yVar` double DEFAULT NULL,\n `centroid_naive_flags` bit(1) NOT NULL,\n `centroid_gaussian_x` double DEFAULT NULL,\n `centroid_gaussian_y` double DEFAULT NULL,\n `centroid_gaussian_xVar` double DEFAULT NULL,\n `centroid_gaussian_xyCov`
double DEFAULT NULL,\n `centroid_gaussian_yVar` double DEFAULT NULL,\n `centroid_gaussian_flags` bit(1) NOT NULL,\n `shape_sdss_Ixx` double DEFAULT NULL,\n `shape_sdss_Iyy` double DEFAULT NULL,\n `shape_sdss_Ixy` double DEFAULT NULL,\n `shape_sdss_IxxVar`
double DEFAULT NULL,\n `shape_sdss_IxxIyyCov` double DEFAULT NULL,\n `shape_sdss_IxxIxyCov` double DEFAULT NULL,\n `shape_sdss_IyyVar` double DEFAULT NULL,\n `shape_sdss_IyyIxyCov` double DEFAULT NULL,\n `shape_sdss_IxyVar` double DEFAULT NULL,\n `shape_sdss_flags`
bit(1) NOT NULL,\n `shape_sdss_centroid_x` double DEFAULT NULL,\n `shape_sdss_centroid_y` double DEFAULT NULL,\n `shape_sdss_centroid_xVar` double DEFAULT NULL,\n `shape_sdss_centroid_xyCov` double DEFAULT NULL,\n `shape_sdss_centroid_yVar` double DEFAULT
NULL,\n `shape_sdss_centroid_flags` bit(1) NOT NULL,\n `shape_sdss_flags_unweightedbad` bit(1) NOT NULL,\n `shape_sdss_flags_unweighted` bit(1) NOT NULL,\n `shape_sdss_flags_shift` bit(1) NOT NULL,\n `shape_sdss_flags_maxiter` bit(1) NOT NULL,\n `flux_psf`
double DEFAULT NULL,\n `flux_psf_err` double DEFAULT NULL,\n `flux_psf_flags` bit(1) NOT NULL,\n `flux_psf_psffactor` float DEFAULT NULL,\n `flux_psf_flags_psffactor` bit(1) NOT NULL,\n `flux_psf_flags_badcorr` bit(1) NOT NULL,\n `flux_naive` double
DEFAULT NULL,\n `flux_naive_err` double DEFAULT NULL,\n `flux_naive_flags` bit(1) NOT NULL,\n `flux_gaussian` double DEFAULT NULL,\n `flux_gaussian_err` double DEFAULT NULL,\n `flux_gaussian_flags` bit(1) NOT NULL,\n `flux_gaussian_psffactor` float DEFAULT
NULL,\n `flux_gaussian_flags_psffactor` bit(1) NOT NULL,\n `flux_gaussian_flags_badcorr` bit(1) NOT NULL,\n `flux_sinc` double DEFAULT NULL,\n `flux_sinc_err` double DEFAULT NULL,\n `flux_sinc_flags` bit(1) NOT NULL,\n `centroid_record_x` double DEFAULT
NULL,\n `centroid_record_y` double DEFAULT NULL,\n `classification_extendedness` double DEFAULT NULL,\n `aperturecorrection` double DEFAULT NULL,\n `aperturecorrection_err` double DEFAULT NULL,\n `refFlux` double DEFAULT NULL,\n `refFlux_err` double
DEFAULT NULL,\n `objectId` bigint(20) NOT NULL,\n `coord_raVar` double DEFAULT NULL,\n `coord_radeclCov` double DEFAULT NULL,\n `coord_declVar` double DEFAULT NULL,\n `exposure_id` bigint(20) NOT NULL,\n `exposure_filter_id` int(11) NOT NULL,\n `exposure_time`
float DEFAULT NULL,\n `exposure_time_mid` double DEFAULT NULL,\n `cluster_id` bigint(20) DEFAULT NULL,\n `cluster_coord_ra` double DEFAULT NULL,\n `cluster_coord_decl` double DEFAULT NULL,\n `chunkId` int(11) NOT NULL,\n `subChunkId` int(11) NOT NULL,\n
PRIMARY KEY (`id`),\n KEY `IDX_parent` (`parent`),\n KEY `IDX_exposure_id` (`exposure_id`),\n KEY `objectId` (`objectId`)\n)"}
/DBS/LSST/TABLES/ForcedSource READY
/DBS/LSST/TABLES/ForcedSource/partitioning.json {"dirTable": "Object", "overlap": "0.0", "latColName": "decl", "lonColName": "ra", "subChunks": "0", "dirColName": "deepSourceId"}
/DBS/LSST/LOCK.json {"comments": "", "reason": "", "mode": "", "estimatedDuration": "", "lockedBy": "", "lockedTime": ""}
/css_meta
/css_meta/version.LOCK
/css_meta/version 1
/PARTITIONING
/PARTITIONING/_0000000000.json {"nStripes": "85", "nSubStripes": "12", "uuid": "baa78d1f-0a13-4f51-9d79-9609aedb64e5", "overlap": "0.01667"}
/PARTITIONING/_0000000000
/zookeeper
/zookeeper/quota
On 11/10/15 14:25, Salnikov, Andrei A. wrote:
There is no easy way to trnser data from zookeeper CSS into mysql,
but I can try to make a simple script to do that. Do you still have
zookeeper running? I need a dump of the old CSS, simplest is to
run qserv-admin from old release (with zookeeper) and say
"DUMP EVERYHTING;". Send the output to me so I can play with that.
Cheers,
Andy
From: Gates, John H
Sent: Tuesday, November 10, 2015 1:44 PM
To: Salnikov, Andrei A.; Fabrice Jammes; qserv-l
Subject: Re: [QSERV-L] Upgrade of IN2P3 css metadata
Hi Andy,
We had to ran this command after yours:
mysql -S /qserv/run/var/lib/mysql/mysql.sock -u root -pchangeme -e "GRANT ALL ON qservCssData.* TO 'qsmaster'@'localhost';"
Configuration files where correct because they have been generated with latest configuration procedure (but not the data which is in2p3 35TB dataset).
But now we have this error message:
ERROR 4110 (Proxy): Query processing error: "NoSuchTable:Table 'LSST.Object' does not exist."
And this error log:
==> /qserv/run/var/log/mysql-proxy-lua.log <==
*******************
Intercepted: select count(*) from Object
proxy.connection.server.thread_id: 7
Passing query: select count(*) from Object
Passing hints: "client_dst_name" "172.17.0.12:4040" "server_thread_id" "7" "db" "LSST"
Sendresult -4110
ERROR errNo: 4110: errMsg: Query processing error: "NoSuchTable:Table 'LSST.Object' does not exist."
==> /qserv/run/var/log/qserv-czar.log <==
2015-11-10T21:33:23.659Z [0x7f90b72ff700] ERROR ccontrol.UserQueryFactory (core/modules/ccontrol/UserQueryFactory.cc:101) - Invalid query: NoSuchTable:Table 'LSST.Object' does not exist.
2015-11-10T21:33:23.659Z [0x7f90b72ff700] INFO root (core/modules/ccontrol/UserQuery.cc:275) - Discarded UserQuery(6)
2015-11-10T21:33:23.662Z [0x7f90b72ff700] ERROR root (appInterface.py:138) - Returning error to proxy: ('"NoSuchTable:Table \'LSST.Object\' does not exist."', False, False, False)
The problem is that we are trying to use in2p3 35TB dataset, and it doesn't have CSS metadata at new format, but zookeeper one. Do you know how we can load this metadata without impacting data please?
This pecore/modules/css/CssAccess.cc:447: if (not _kvI->exists(tableKey)) throw NoSuchTable(dbName, tableName);
core/modules/css/CssAccess.cc:482: if (not _kvI->exists(tableKey)) throw NoSuchTable(dbName, tableName);
core/modules/css/CssAccess.cc:501: if (not _kvI->exists(tableKey)) throw NoSuchTable(dbName, tableName);
core/modules/css/CssAccess.cc:531: if (not _kvI->exists(tableKey)) throw NoSuchTable(dbName, tableName);
core/modules/css/CssAccess.cc:577: if (not _kvI->exists(tableKey)) throw NoSuchTable(dbName, tableName);
core/modules/css/CssAccess.cc:712: throw NoSuchTable(dbName, tableName);
core/modules/css/CssAccess.cc:902: if (not _kvI->exists(tableKey)) throw NoSuchTable(dbName, tableName);ace of code seems to be looking for CSS data which doesn't exist:
Thanks :-)
On 11/10/15 11:58, Salnikov, Andrei A. wrote:
Hi Fabrice,
it looks likely due to missing database for CSS data.
Did you swich to the latest qserv release already? Did
you run qserv-configure when you did that?
If you do not want to run qserv-configure with a new release
you need to do some manual work for updating your setup:
- create database and tables for CSS:
$ mysql -S /path/to/socket/file -uroot -p*** < admin/templates/configuration/tmp/configure/sql/CssData.sql
- update czar config file with CSS options, section [css]
should look like:
[css]
technology=mysql
hostname =
port =
username = qsmaster
password =
database = qservCssData
socket = /czar_instance_run_dir/var/lib/mysql/mysql.sock
- restart czar
Cheers,
Andy
From: [log in to unmask] [mailto:[log in to unmask]] On Behalf Of Fabrice Jammes
Sent: Tuesday, November 10, 2015 9:42 AM
To: qserv-l
Subject: [QSERV-L] Upgrade of IN2P3 css metadata
Hello,
CSS metadata is still stored in zookeeper on cc-in2p3 cluster, so any SQL query on a partitioned table crashes Qserv.
Does somebody knows how to upgrade css metadata to new format?
If it is proven that the crash is due to wrong metadata format, I'll open a bug, this shouldn't crash Qserv but print explicit log message instead, isn't i?
Error report is available below.
Thanks,
-------- Forwarded Message --------
Subject:
About docker install
Date:
Mon, 09 Nov 2015 16:58:23 -0800
From:
Fabrice Jammes <[log in to unmask]>
To:
John Gates <[log in to unmask]>
Hi John,It seems our docker configuration is working a bit:fjammes@ccosvms0070:~$ time mysql --host ccqserv125 --port 4040 --userqsmaster LSST -e "show tables;";+------------------------------------+| Tables_in_LSST |+------------------------------------+| ForcedSource || ForcedSourceFullOverlap_1234567890 || ForcedSource_1234567890 || Object || ObjectFullOverlap_1234567890 || Object_1234567890 || Source || SourceFullOverlap_1234567890 || Source_1234567890 |+------------------------------------+real 0m0.027suser 0m0.010ssys 0m0.013sNevertheless next query fails and crash qserv czar:fjammes@ccosvms0070:~$ time mysql --host ccqserv125 --port 4040 --userqsmaster LSST -e "select count(*) from Object;";ERROR 4005 (Proxy) at line 1: Unable to run lua xmlrpc client, message:...inux64/lua/5.1.4.lsst1/share/lua/5.1/xmlrpc/http.lua:41: closedreal 0m0.105suser 0m0.004ssys 0m0.009sqserv-czar.log doesn't print any error message:qserv@ccqserv125:/qserv$ tail /qserv/run/var/log/qserv-czar.loguser = qsmasterport = 02015-11-10T00:53:52.413Z [0x7f98d86af700] DEBUG root(appInterface.py:70) - _idCounter 279968324122015-11-10T00:53:52.413Z [0x7f98d86af700] INFO root (server.py:137) -endpoints: [<bound method AppInterface.cancelEverything of<lsst.qserv.czar.appInterface.AppInterface instance at 0x7f98cba83878>>,<bound method AppInterface.check of<lsst.qserv.czar.appInterface.AppInterface instance at 0x7f98cba83878>>,<bound method AppInterface.joinQuery of<lsst.qserv.czar.appInterface.AppInterface instance at 0x7f98cba83878>>,<bound method AppInterface.killQuery of<lsst.qserv.czar.appInterface.AppInterface instance at 0x7f98cba83878>>,<bound method AppInterface.killQueryUgly of<lsst.qserv.czar.appInterface.AppInterface instance at 0x7f98cba83878>>,<bound method AppInterface.resultTableString of<lsst.qserv.czar.appInterface.AppInterface instance at 0x7f98cba83878>>,<bound method AppInterface.results of<lsst.qserv.czar.appInterface.AppInterface instance at 0x7f98cba83878>>,<bound method AppInterface.submitQuery of<lsst.qserv.czar.appInterface.AppInterface instance at 0x7f98cba83878>>,<bound method Czar.echo of <lsst.qserv.czar.server.Czar instance at0x7f98cba83830>>, <bound method Czar.stop of<lsst.qserv.czar.server.Czar instance at 0x7f98cba83830>>, <bound methodCzar.reset of <lsst.qserv.czar.server.Czar instance at 0x7f98cba83830>>]2015-11-10T00:53:52.420Z [0x7f98d86af700] INFO root (server.py:128) -Starting Qserv interface on port: 70802015-11-10T00:56:40.602Z [0x7f98d86af700] DEBUG root(appInterface.py:119) - taskId 279968324122015-11-10T00:56:40.613Z [0x7f98d86af700] DEBUG css.CssAccess(core/modules/css/CssAccess.cc:95) - Create CSS instance from config map2015-11-10T00:56:40.613Z [0x7f98d86af700] DEBUG css.CssAccess(core/modules/css/CssAccess.cc:148) - Create CSS instance with mysql store
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1