 |
 |
I added this test to test code that breaks worker responses into reasonably sized chunks. I'd like to help sort out what's going wrong, but I'm not sure what the best way to go about it. -John On 05/07/15 11:10, Vaikunth Thukral wrote: > Hi all, > > So I've been able to resolve the remaining issues with the data loader > (thanks AndyS for your help) for integration tests in multi-node and > cases 1, 2, 3 and 5 are now working. For case 4 however the last query > fails (query 0030_largeResult.sql). The czar log gives the following > (full logs available on dbdev1 at > /usr/local/home/vaikunth/qserv-run/var/log/): > > 0507 12:45:37.872 [0x7f7cf2137700] INFO root > (build/rproc/InfileMerger.cc:351) - InfileMerger 0x7f7cdc0105b0 > connected to db. > 0507 12:45:37.872 [0x7f7cf2137700] ERROR root > (build/rproc/InfileMerger.cc:359) - InfileMerger sql error: Error > applying sql. Error 1146: Tabl\ > e 'qservResult.result_11900632096_m' doesn't exist Unable to execute > query: CREATE TABLE qservResult.result_11900632096 SELECT deepForcedSourc\ > eId,scienceCcdExposureId,filterId,deepSourceId,timeMid,expTime,ra,decl,raVar,declVar,radeclCov,htmId20,x,y,xVar,yVar,xyCov,psfFlux,psfFluxSigm\ > a,apFlux,apFluxSigma,modelFlux,modelFluxSigma,instFlux,instFluxSigma,apCorrection,apCorrectionSigma,shapeIx,shapeIy,shapeIxVar,shapeIyVar,shap\ > eIxIyCov,shapeIxx,shapeIyy,shapeIxy,shapeIxxVar,shapeIyyVar,shapeIxyVar,shapeIxxIyyCov,shapeIxxIxyCov,shapeIyyIxyCov,extendedness,flagNegative\ > ,flagBadMeasCentroid,flagPixEdge,flagPixInterpAny,flagPixInterpCen,flagPixSaturAny,flagPixSaturCen,flagBadPsfFlux,flagBadApFlux,flagBadModelFl\ > ux,flagBadInstFlux,flagBadCentroid,flagBadShape,raDeepSource,declDeepSource > FROM qservResult.result_11900632096_m ORDER BY deepForcedSourceId > > 0507 12:45:37.872 [0x7f7cf2137700] INFO root > (build/rproc/InfileMerger.cc:313) - Cleaning up > qservResult.result_11900632096_m > 0507 12:45:37.872 [0x7f7cf2137700] INFO root > (build/rproc/InfileMerger.cc:325) - Merged > qservResult.result_11900632096_m into qservResult.res\ > ult_11900632096 > 0507 12:45:37.872 [0x7f7cf2137700] ERROR root > (build/ccontrol/UserQuery.cc:206) - Joined everything (failure!) > 0507 12:45:37.874 [0x7f7cf2137700] INFO root (app.py:408) - Query > exec (5) took 1.873482 seconds > 0507 12:45:37.874 [0x7f7cf2137700] ERROR root > (build/qdisp/Executive.cc:360) - Ref=1 > Resource(/chk/qservTest_case04_qserv/6970): 20150507-12:4\ > 5:37, Error in result data., 1, > Ref=2 Resource(/chk/qservTest_case04_qserv/7138): 20150507-12:45:37, > Error in result data., 1, > Ref=3 Resource(/chk/qservTest_case04_qserv/7140): 20150507-12:45:37, > Error in result data., 1, > Ref=4 Resource(/chk/qservTest_case04_qserv/7308): 20150507-12:45:37, > Error in result data., 1, > Ref=5 Resource(/chk/qservTest_case04_qserv/7310): 20150507-12:45:37, > Error in result data., 1, > 0507 12:45:37.875 [0x7f7cf2137700] DEBUG root (app.py:343) - reporting > -1 -1 Ref=1 Resource(/chk/qservTest_case04_qserv/6970): 20150507-12:45:\ > 37, Error in result data., 1, > Ref=2 Resource(/chk/qservTest_case04_qserv/7138): 20150507-12:45:37, > Error in result data., 1, > Ref=3 Resource(/chk/qservTest_case04_qserv/7140): 20150507-12:45:37, > Error in result data., 1, > Ref=4 Resource(/chk/qservTest_case04_qserv/7308): 20150507-12:45:37, > Error in result data., 1, > Ref=5 Resource(/chk/qservTest_case04_qserv/7310): 20150507-12:45:37, > Error in result data., 1, > 0507 12:45:37.875 [0x7f7cf2137700] ERROR root > (build/qdisp/MessageStore.cc:47) - Msg: -1 -1 Ref=1 > Resource(/chk/qservTest_case04_qserv/6970): \ > 20150507-12:45:37, Error in result data., 1, > Ref=2 Resource(/chk/qservTest_case04_qserv/7138): 20150507-12:45:37, > Error in result data., 1, > Ref=3 Resource(/chk/qservTest_case04_qserv/7140): 20150507-12:45:37, > Error in result data., 1, > Ref=4 Resource(/chk/qservTest_case04_qserv/7308): 20150507-12:45:37, > Error in result data., 1, > Ref=5 Resource(/chk/qservTest_case04_qserv/7310): 20150507-12:45:37, > Error in result data., 1, > 0507 12:45:37.876 [0x7f7cf2137700] INFO root (app.py:413) - Final > state of all queries error > /usr/local/home/vaikunth/src/qserv/lib/python/lsst/qserv/czar/db.py:100: > Warning: Data truncated for column 'message' at row 1 > c.execute(sql, params) > 0507 12:45:37.878 [0x7f7cf1736700] INFO root > (build/ccontrol/UserQuery.cc:236) - Discarded UserQuery(5) > > I am not sure if this is related to what Fabrice is seeing at in2p3, > but when I run a simpler version of the 0030_ query myself I get (with > error log also in the same location as mentioned above): > > [vaikunth@lsst-dbdev1 qserv_testdata]$ mysql > --host=lsst-dbdev1.ncsa.illinois.edu > <http://lsst-dbdev1.ncsa.illinois.edu> --port=4040 --user=qsmaster > --batch qservTest_case04_qserv -e "SELECT * FROM DeepForcedSource > ORDER BY deepForcedSourceId" > ERROR 4120 (Proxy) at line 1: Error during execution: > -1 Ref=1 Resource(/chk/qservTest_case04_qserv/6970): > 20150507-13:07:32, Error in result data., 1, > Ref=2 Resource(/chk/qservTest_case04_qserv/7138): 20150507-13:07:32, > Error in result data., 1, > Ref=3 Resource(/chk/qservTest_case04_qserv/7140): 20150507-13:0 (-1) > > -Vaikunth > > ------------------------------------------------------------------------ > > Use REPLY-ALL to reply to list > > To unsubscribe from the QSERV-L list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1 > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the QSERV-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1