Hi Bill,
Good job and good news ! Your install now works. It's not so easy to set
up a working qserv instance, isn't it ;-)
On 26/11/2013 21:13, Bill Chickering wrote:
> Hi Fabrice -
>
> Okay, my mistake. I now have a working instance of master branch
> qserv--thanks!
>
> I did encounter a couple of minor snags, however. Perhaps these are
> unique to my setup or due to errors on my part, but I mention them for
> posterity. . .
>
> As part of my installation, I migrated data from another qserv
> instance. To do this, I merely copied %QSERV_BASE_DIR%/var/lib/mysql
> from the preexisting instance to the new instance. Having migrating
> metadata, qserv now successfully starts upon issuing `qserv-start`.
Copying %QSERV_BASE_DIR%/var/lib/mysql also copy
%QSERV_BASE_DIR%/var/lib/mysql/mysql which contains administrative
informations,of whose privileges and access rights. This may cause the
next problem on qservResult database.
This kind of data loading is mastered by Douglas.
>
> At this point query execution fails, however, due to insufficient
> permissions on the qservResult database (I see errors in the qserv log
> saying ''@'localhost' cannot access qservResult). To remove this
> obstacle, I connect to the qservResult database as root and issue the
> command `grant all privileges on *.* to ''@'localhost' with grant
> option;`. Certainly this is not ideal, but it solved my immediate issue.
>
> At this point query execution still fails due to the absence of
> %QSERV_BASE_DIR%/etc/emptyChunks.txt. Copying this file over from the
> preexisting instance solves the problem. I now have a working instance
> of qserv.
The test procedure create %QSERV_BASE_DIR%/etc/emptyChunks.txt for each
test dataset (cf. qservdataloader.py). In case you load an other
dataset, you also have to load its meta (and not only the QMS ones), of
whose %QSERV_BASE_DIR%/etc/emptyChunks.txt file. All this is managed by
Douglas scripts for several dataset (cf. qserv-chunkload.py and
qserv-admin.pl) and should be covered by the future unified dataloading
procedure.
>
> While qserv seems to be essentially working correctly, here's what I
> encounter upon issuing `qserv-start` and `qserv-stop`:
>
> -bash-4.1$ qserv-start
> chown: invalid group: `bchick:bchick'
> chown: invalid group: `bchick:bchick'
> Starting mysqld: [ OK ]
> Starting mysql-proxy: [ OK ]
> Starting xrootd: [ OK ]
> Starting qms: [ OK ]
> Starting qserv-master: [ OK ]
> -bash-4.1$ qserv-stop
> Stopping mysqld: [ OK ]
> Stopping mysql-proxy: [FAILED]
> Stopping xrootd: [ OK ]
> Stopping qms: [ OK ]
> Stopping qserv-master: [ OK ]
> -bash-4.1$
>
> Note that all qserv-related processes have indeed stopped after
> issuing `qserv-stop`.
Thanks for these very interesting informations which are worth opening a
Trac ticket about init.d script.
The chown warning is issued by mysqld init.d script because you don't
belongs to group bchick. This init.d script is nearly the same than the
one of SL6.2 distribution. This problem is non blocking and can be
easily solved.
For the mysql-proxy issue, i need a more detailed diagnostic in order to
solve it. Could you please find interesting things in the log ? is
mysql-proxy pid file always here ? ...
The services should also be stopped in reverse order.
I think it's important to have robusts init.d scripts, but this require
putting some stress on them with multiple user tests.
Thanks and have a nice day,
Fabrice
> Anyhow, I'm happy to have a running instance of the master branch!
> Thanks for your help!
>
> Regards,
> Bill
>
> On Nov 26, 2013, at 12:21 AM, Fabrice Jammes <[log in to unmask]
> <mailto:[log in to unmask]>> wrote:
>
>> Hello Bill,
>>
>> I can't see the script start_xrootd, start_qms, or start_qserv
>> mentionned inhttps://dev.lsstcorp.org/trac/wiki/db/Qserv/InstallAuto
>> <https://dev.lsstcorp.org/trac/wiki/db/Qserv/InstallAuto>
>> are you reading the latest version of this page ?
>>
>> Indeed, start_* scripts have been replaced by more standard init.d
>> script since #3013.
>>
>> qserv-start and qserv-stop are still used as wrapper to start/stop
>> all qserv services, but you've first to source qserv-env.sh, in order
>> to enable these 2 aliases.
>> If you want to know more about Qserv services fine grained
>> management, please have a look at :
>> https://dev.lsstcorp.org/trac/wiki/db/Qserv/initd
>>
>> Hope this help.
>>
>> Fabrice
>>
>>
>>
>> On 11/25/2013 10:55 PM, Bill Chickering wrote:
>>> Hi Fabrice -
>>>
>>> Thanks for merging #3006. I've pulled the latest master branch and
>>> am attempting to build a single node instance of qserv from scratch.
>>> It appears that the dependencies were successfully built. Here's the
>>> tail of the output I observe at the terminal after running `scons`
>>> within the src directory:
>>>
>>> DEBUG : {'XRD_DIR': '/u1/bchick/sandbox2/qserv-master/xrootd',
>>> 'PROTOC': '/u1/bchick/sandbox2/qserv-master/bin/protoc',
>>> 'PROTOC_INC': '/u1/bchick/sandbox2/qserv-master/include',
>>> 'QSW_DBSOCK':
>>> '/u1/bchick/sandbox2/qserv-master/var/lib/mysql/mysql.sock',
>>> 'MYSQL_ROOT': '/u1/bchick/sandbox2/qserv-master', 'SHLVL': '2',
>>> 'OLDPWD': '/u1/bchick/sandbox2/qserv-master/qserv/master',
>>> 'SEARCH_ROOTS': '/u1/bchick/sandbox2/qserv-master', 'PWD':
>>> '/u1/bchick/sandbox2/qserv-master/qserv/worker', 'XRD_PLATFORM':
>>> 'x86_64_linux_26_dbg', 'PATH':
>>> '/usr/local/bin:/opt/bin:/bin:/usr/bin', 'LD_LIBRARY_PATH':
>>> '/u1/bchick/sandbox2/qserv-master/lib:/usr/lib64:/u1/bchick/sandbox2/qserv-master/lib/mysql:/u1/bchick/sandbox2/qserv-master/lib:/usr/lib64:/u1/bchick/sandbox2/qserv-master/qserv/worker/../common/lib:/u1/bchick/sandbox2/qserv-master/qserv/worker/../common/lib:/u1/bchick/sandbox2/qserv-master/lib:/u1/bchick/sandbox2/qserv-master/lib/mysql:/u1/bchick/sandbox2/qserv-master/qserv/worker/bldNoXrd',
>>> '_': '/usr/bin/scons', 'PROTOC_LIB':
>>> '/u1/bchick/sandbox2/qserv-master/lib'}
>>> g++ -o tests/testSanityCheck.o -c -g -DNO_XROOTD_FS
>>> -I/u1/bchick/sandbox2/qserv-master/include/xrootd
>>> -I/u1/bchick/sandbox2/qserv-master/include/xrootd -Iinclude
>>> -I/u1/bchick/src/qserv-master/common/include
>>> -I/u1/bchick/sandbox2/qserv-master/include tests/testSanityCheck.cc
>>> <http://testsanitycheck.cc/>
>>> g++ -o tests/testSanityCheck tests/testSanityCheck.o
>>> bldNoXrd/lib/libqserv_worker.so bldNoXrd/lib/libqserv_workerCmsd.so
>>> -L/u1/bchick/sandbox2/qserv-master/lib -L/usr/lib64
>>> -L/u1/bchick/sandbox2/qserv-master/lib/mysql
>>> -L/u1/bchick/sandbox2/qserv-master/lib -L/usr/lib64
>>> -L/u1/bchick/src/qserv-master/common/lib
>>> -L/u1/bchick/src/qserv-master/common/lib
>>> -L/u1/bchick/sandbox2/qserv-master/lib
>>> -L/u1/bchick/sandbox2/qserv-master/lib/mysql -LbldNoXrd -lssl
>>> -lcrypto -lboost_regex-mt -lboost_thread-mt -lboost_signals-mt
>>> -lprotobuf -lssl -lcrypto -lmysqlclient_r -lmysqlclient_r
>>> -lqserv_common -lqserv_proto -lXrdUtils -lXrdClient -lXrdServer
>>> -lXrdSys -lXrdOfs -lXrdSfs -lmysqlclient_r -lXrdSys -lXrdClient
>>> -lXrdUtils
>>> run_test(["tests/.tests/testSanityCheck"], ["tests/testSanityCheck"])
>>> DEBUG : {'XRD_DIR': '/u1/bchick/sandbox2/qserv-master/xrootd',
>>> 'PROTOC': '/u1/bchick/sandbox2/qserv-master/bin/protoc',
>>> 'PROTOC_INC': '/u1/bchick/sandbox2/qserv-master/include',
>>> 'QSW_DBSOCK':
>>> '/u1/bchick/sandbox2/qserv-master/var/lib/mysql/mysql.sock',
>>> 'MYSQL_ROOT': '/u1/bchick/sandbox2/qserv-master', 'SHLVL': '2',
>>> 'OLDPWD': '/u1/bchick/sandbox2/qserv-master/qserv/master',
>>> 'SEARCH_ROOTS': '/u1/bchick/sandbox2/qserv-master', 'PWD':
>>> '/u1/bchick/sandbox2/qserv-master/qserv/worker', 'XRD_PLATFORM':
>>> 'x86_64_linux_26_dbg', 'PATH':
>>> '/usr/local/bin:/opt/bin:/bin:/usr/bin', 'LD_LIBRARY_PATH':
>>> '/u1/bchick/sandbox2/qserv-master/lib:/usr/lib64:/u1/bchick/sandbox2/qserv-master/lib/mysql:/u1/bchick/sandbox2/qserv-master/lib:/usr/lib64:/u1/bchick/sandbox2/qserv-master/qserv/worker/../common/lib:/u1/bchick/sandbox2/qserv-master/qserv/worker/../common/lib:/u1/bchick/sandbox2/qserv-master/lib:/u1/bchick/sandbox2/qserv-master/lib/mysql:/u1/bchick/sandbox2/qserv-master/qserv/worker/bldNoXrd',
>>> '_': '/usr/bin/scons', 'PROTOC_LIB':
>>> '/u1/bchick/sandbox2/qserv-master/lib'}
>>> g++ -o tests/PathStructure_1.o -c -g -DNO_XROOTD_FS
>>> -I/u1/bchick/sandbox2/qserv-master/include/xrootd
>>> -I/u1/bchick/sandbox2/qserv-master/include/xrootd -Iinclude
>>> -I/u1/bchick/src/qserv-master/common/include
>>> -I/u1/bchick/sandbox2/qserv-master/include tests/PathStructure_1.cc
>>> g++ -o tests/PathStructure_1 tests/PathStructure_1.o
>>> src/QservPathStructure.o -L/u1/bchick/sandbox2/qserv-master/lib
>>> -L/usr/lib64 -L/u1/bchick/sandbox2/qserv-master/lib/mysql
>>> -L/u1/bchick/sandbox2/qserv-master/lib -L/usr/lib64
>>> -L/u1/bchick/src/qserv-master/common/lib
>>> -L/u1/bchick/src/qserv-master/common/lib
>>> -L/u1/bchick/sandbox2/qserv-master/lib
>>> -L/u1/bchick/sandbox2/qserv-master/lib/mysql -LbldNoXrd
>>> scons: done building targets.
>>> Stopping mysqld: [ OK ]
>>> scons: done building targets.
>>>
>>>
>>> And here is the tail of INSTALL-2013-11-25-12:36:46.log:
>>>
>>> make[1]: Leaving directory
>>> `/u1/bchick/sandbox2/qserv-master/build/mysql-proxy-0.8.2/doc'
>>> make[1]: Entering directory
>>> `/u1/bchick/sandbox2/qserv-master/build/mysql-proxy-0.8.2'
>>> make[2]: Entering directory
>>> `/u1/bchick/sandbox2/qserv-master/build/mysql-proxy-0.8.2'
>>> make[2]: Nothing to be done for `install-exec-am'.
>>> test -z "/u1/bchick/sandbox2/qserv-master/lib/pkgconfig" ||
>>> /bin/mkdir -p "/u1/bchick/sandbox2/qserv-master/lib/pkgconfig"
>>> /usr/bin/install -c -m 644 'mysql-proxy.pc'
>>> '/u1/bchick/sandbox2/qserv-master/lib/pkgconfig/mysql-proxy.pc'
>>> /usr/bin/install -c -m 644 'mysql-chassis.pc'
>>> '/u1/bchick/sandbox2/qserv-master/lib/pkgconfig/mysql-chassis.pc'
>>> make[2]: Leaving directory
>>> `/u1/bchick/sandbox2/qserv-master/build/mysql-proxy-0.8.2'
>>> make[1]: Leaving directory
>>> `/u1/bchick/sandbox2/qserv-master/build/mysql-proxy-0.8.2'
>>> QSERV DEPENDENCIES SUCCESSFULLY INSTALLED
>>>
>>> Within the qserv install directory, I now see the start_mysqlproxy
>>> script. I do not, however, see start_xrootd, start_qms, or
>>> start_qserv. These scripts should have been created during the
>>> install, correct? Did I make mistake or skip a step? (btw, I have
>>> read https://dev.lsstcorp.org/trac/wiki/db/Qserv/InstallAuto and
>>> tried to follow the procedure outlined there).
>>>
>>> Thanks,
>>> Bill
>>>
>>>
>>> On Nov 25, 2013, at 4:48 AM, Fabrice Jammes <[log in to unmask]
>>> <mailto:[log in to unmask]>> wrote:
>>>
>>>> Hello Bill,
>>>>
>>>> #3006 is now merged to master.
>>>>
>>>> Have fun!
>>>>
>>>> Fabrice
>>>>
>>>> On 11/25/2013 09:29 AM, Bill Chickering wrote:
>>>>> Sounds good. Thanks, Fabrice.
>>>>>
>>>>>
>>>>> On Nov 25, 2013, at 12:23 AM, Fabrice Jammes
>>>>> <[log in to unmask] <mailto:[log in to unmask]>> wrote:
>>>>>
>>>>>> Hello Bill,
>>>>>>
>>>>>> Ticket #3006 solves this issue. It as just been reviewed on
>>>>>> Friday, i'll merge it soon and let you know.
>>>>>>
>>>>>> Fabrice
>>>>>>
>>>>>>
>>>>>> On 11/24/2013 11:57 PM, Bill Chickering wrote:
>>>>>>> Hi Guys -
>>>>>>>
>>>>>>> I'm trying to build the master branch qserv from scratch (this
>>>>>>> is the master before merging 3019 and 3020). Running into the
>>>>>>> same problem I mentioned to Douglas and Fabrice during the
>>>>>>> hackathon. I thought that Fabrice said a ticket was merged that
>>>>>>> resolved this issue, but evidently that's not the case.
>>>>>>> Here's the error I encounter:
>>>>>>>
>>>>>>> run_test(["tests/.tests/QuerySql_1"], ["tests/QuerySql_1"])
>>>>>>> g++ -o tests/QueueTest.o -c -g -DNO_XROOTD_FS
>>>>>>> -I/u1/bchick/sandbox2/qserv-master/include/xrootd
>>>>>>> -I/u1/bchick/sandbox2/qserv-master/include/xrootd -Iinclude
>>>>>>> -I/u1/bchick/src/qserv-master/common/include -
>>>>>>> I/u1/bchick/sandbox2/qserv-master/include tests/QueueTest.cc
>>>>>>> <http://queuetest.cc/>
>>>>>>> g++ -o tests/QueueTest tests/QueueTest.o
>>>>>>> -L/u1/bchick/sandbox2/qserv-master/lib -L/usr/lib64
>>>>>>> -L/u1/bchick/sandbox2/qserv-master/lib/mysql
>>>>>>> -L/u1/bchick/sandbox2/qserv-master/lib -L/usr/lib64 -L/u1/bchick
>>>>>>> /src/qserv-master/common/lib
>>>>>>> -L/u1/bchick/src/qserv-master/common/lib
>>>>>>> -L/u1/bchick/sandbox2/qserv-master/lib
>>>>>>> -L/u1/bchick/sandbox2/qserv-master/lib/mysql -LbldNoXrd
>>>>>>> run_test(["tests/.tests/QueueTest"], ["tests/QueueTest"])
>>>>>>> g++ -o tests/testSanityCheck.o -c -g -DNO_XROOTD_FS
>>>>>>> -I/u1/bchick/sandbox2/qserv-master/include/xrootd
>>>>>>> -I/u1/bchick/sandbox2/qserv-master/include/xrootd -Iinclude
>>>>>>> -I/u1/bchick/src/qserv-master/common/inc
>>>>>>> lude -I/u1/bchick/sandbox2/qserv-master/include
>>>>>>> tests/testSanityCheck.cc <http://testsanitycheck.cc/>
>>>>>>> g++ -o tests/testSanityCheck tests/testSanityCheck.o
>>>>>>> bldNoXrd/lib/libqserv_worker.so
>>>>>>> bldNoXrd/lib/libqserv_workerCmsd.so
>>>>>>> -L/u1/bchick/sandbox2/qserv-master/lib -L/usr/lib64
>>>>>>> -L/u1/bchick/sandbox2/qserv-m
>>>>>>> aster/lib/mysql -L/u1/bchick/sandbox2/qserv-master/lib
>>>>>>> -L/usr/lib64 -L/u1/bchick/src/qserv-master/common/lib
>>>>>>> -L/u1/bchick/src/qserv-master/common/lib
>>>>>>> -L/u1/bchick/sandbox2/qserv-master/lib -L/u1/bchick/
>>>>>>> sandbox2/qserv-master/lib/mysql -LbldNoXrd -lssl -lcrypto
>>>>>>> -lboost_regex-mt -lboost_thread-mt -lboost_signals-mt -lprotobuf
>>>>>>> -lssl -lcrypto -lmysqlclient_r -lmysqlclient_r -lqserv_common
>>>>>>> -lqserv_proto -lX
>>>>>>> rdUtils -lXrdClient -lXrdServer -lXrdSys -lXrdOfs -lXrdSfs
>>>>>>> -lmysqlclient_r -lXrdSys -lXrdClient -lXrdUtils
>>>>>>> run_test(["tests/.tests/testSanityCheck"],
>>>>>>> ["tests/testSanityCheck"])
>>>>>>> *** TEST FAILED *** tests/testSanityCheck 201
>>>>>>> scons: *** [tests/.tests/testSanityCheck] Error 201
>>>>>>> scons: building terminated because of errors.
>>>>>>>
>>>>>>> Any suggestions?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Bill
>>>>>>>
>>>>>>>
>>>>>>> On Nov 22, 2013, at 7:44 PM, Jacek Becla
>>>>>>> <[log in to unmask] <mailto:[log in to unmask]>> wrote:
>>>>>>>
>>>>>>>> Bill,
>>>>>>>>
>>>>>>>> If you need any help with merging your tickets to the master,
>>>>>>>> please let me know.
>>>>>>>>
>>>>>>>> The page I mentioned at the meeting that might help:
>>>>>>>>
>>>>>>>> https://dev.lsstcorp.org/trac/wiki/GitDemoAndTutorial#LSSTgitworkflowandbranchmanagementpolicy
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Jacek
>>>>>>> ########################################################################
>>>>>>> Use REPLY-ALL to reply to list
>>>>>>>
>>>>>>> To unsubscribe from the QSERV-L list, click the following link:
>>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>>>>>> ########################################################################
>>>>>> Use REPLY-ALL to reply to list
>>>>>>
>>>>>> To unsubscribe from the QSERV-L list, click the following link:
>>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>>>>> ########################################################################
>>>>> Use REPLY-ALL to reply to list
>>>>>
>>>>> To unsubscribe from the QSERV-L list, click the following link:
>>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>>>>
>>>> ########################################################################
>>>> Use REPLY-ALL to reply to list
>>>>
>>>> To unsubscribe from the QSERV-L list, click the following link:
>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> Use REPLY-ALL to reply to list
>>>
>>> To unsubscribe from the QSERV-L list, click the following link:
>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the QSERV-L list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
>>
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1
