Excellent! Thanks, Douglas! I'll give this a shot later this afternoon and let you know how it goes.
I'm more than happy to be the guinea pig for setting up test data if that means building/installing/configuring qserv becomes a well established process.
-- Bill

On Sep 4, 2013, at 12:57 PM, Douglas Smith <[log in to unmask]> wrote:

> 
> Ah, sorry, didn't mention the data is on lsst-dev03 for you there.
> 
> Douglas
> 
> 
> On 09/04/2013 12:56 PM, Douglas Smith wrote:
>> Well, not sure I am helping at this point, but maybe?
>> 
>> But here Bill, I created a newly partitioned pt11 dataset.  You can get
>> this at:
>> 
>> /u1/douglas/testing/data02/pt11_part
>> 
>> And there I put a qms_setup_pt11.sh script to create meta data
>> that should work with this data. Just try and load the Object tables,
>> and let me know if that maybe should work?
>> 
>> Also I should take this further and supply an emptyChunks.txt that
>> will work for this, and also maybe just the already loaded myisam files
>> so you can just copy them in.
>> 
>> But it might be good to do that for all, or most test data that we
>> have?  Bill could be a beta tester on that, so we can setup a place
>> for the test data, and scripts to create meta data databases ready
>> to use.
>> 
>> Douglas
>> 
>> 
>> On 09/04/2013 12:17 AM, Bill Chickering wrote:
>>> Hi Daniel -
>>> 
>>> I believe you're recommending that I set up data replication using your mysqld on lsst-dev01 as the master and mine on lsst-dev03 as a slave. I don't believe this is trivial to set up (although, it doesn't look too hard):
>>> http://dev.mysql.com/doc/refman/5.0/en/replication-howto.html
>>> 
>>> Is there an obvious reason why I shouldn't simply copy the data over? Because, that's what I did. . .
>>> 
>>> I copied all object data over using mysqldump. I also copied the two qms databases. qserv and qms seem to startup okay, but I receive the following error in response to a simple query:
>>> mysql> select * from Object limit 1;
>>> ERROR 4120 (Proxy): Error during execution: 'open failed for chunk(s): 1234567890'
>>> 
>>> Any suggestions?
>>> 
>>> Also, what does your empyChunks.txt look like? I believe I need a copy of this.
>>> 
>>> Thanks,
>>> Bill
>>> 
>>> 
>>> On Sep 3, 2013, at 3:13 PM, "Daniel L. Wang" <[log in to unmask]> wrote:
>>> 
>>>> Just get a mirror of the LSST database (for the data) and qms_qms1 qms_qms1_LSST databases from the mysqld accessible via /u1/local/mysql.sock .
>>>> 
>>>> Once you've populated your own mysqld, then you should be able to restart the qms (it should pick up the new metadata) and the qserv frontend (it should pull the right setttings from the restarted qms).
>>>> 
>>>> -Daniel
>>>> 
>>>> On 09/03/2013 03:07 PM, Bill Chickering wrote:
>>>>> Great, thanks! Where, specifically, is the data located? I would like to load this data into my dev instance of qserv, using the same partitioning parameters you use.
>>>>> -- Bill
>>>>> 
>>>>> On Sep 3, 2013, at 3:01 PM, "Daniel L. Wang" <[log in to unmask]> wrote:
>>>>> 
>>>>>> You are welcome to use the pt11 data that I am using on lsst-dev01.
>>>>>> 
>>>>>> The old partitioner still works, and its performance is adequate at pt11 scale.
>>>>>> 
>>>>>> -Daniel
>>>>>> 
>>>>>> On 09/03/2013 11:58 AM, Bill Chickering wrote:
>>>>>>> Thanks, Douglas. Unfortunately, neither 60 stripes, 20 substripes nor 85 stripes, 12 substripes seems to work. Do the contents of ObjectPartitions.csv provide any clues?:
>>>>>>> 100,0,1628411,0.0,9.47368421053,0.0,9.0,0.01667,0.0168777936529
>>>>>>> 118,1,379476,350.526315789,360.0,0.0,9.0,0.01667,0.0168777936529
>>>>>>> 80,5,1631301,0.0,9.47368421053,-9.0,0.0,0.01667,0.0168777936529
>>>>>>> 98,6,373153,350.526315789,360.0,-9.0,0.0,0.01667,0.0168777936529
>>>>>>> 
>>>>>>> Meanwhile, if anyone can direct me to an alternative dataset that is partitioned with known parameters, I might be able to work with that. Otherwise, I am unable to work with the latest versions of qserv that incorporate qms.
>>>>>>> 
>>>>>>> -- Bill
>>>>>>> 
>>>>>>> 
>>>>>>> On Sep 3, 2013, at 8:41 AM, Douglas Smith <[log in to unmask]> wrote:
>>>>>>> 
>>>>>>>> On 09/02/2013 05:16 PM, Bill Chickering wrote:
>>>>>>>>> Dear Qserv Team -
>>>>>>>>> 
>>>>>>>>> I have yet to resolve my metadata/partitioning issues. I am not able to proceed without at least one of the following:
>>>>>>>>> 
>>>>>>>>> 1) Repair my present instance of qserv, which uses pt11 object data from /u1/douglas/testing/data. This is broken because I do not know the correct options to provide qms. Perhaps Douglas can provide this information.
>>>>>>>> Ok, I can't find the exact details on how I made this data, it was a while ago
>>>>>>>> there.  But it was either 60 stripes, 20 substripes, or 85 stripes, 12 substripes.
>>>>>>>> Most of the partition testing I've done as been with 85 stripes, 12 substripes.
>>>>>>>> But there was a very early test that I did with 60 stripes and 20 substripes,
>>>>>>>> and that might be this data.
>>>>>>>> 
>>>>>>>> You can try those options, and see how that goes.
>>>>>>>> 
>>>>>>>>> 2) Alternative partitioned data with known qms options. Daniel, I believe you mentioned you could provide this.
>>>>>>>>> 
>>>>>>>>> 3) I could partition data myself. To do this, I believe I need the new partitioner. The Trac pages say the source for this can be found at qserv/admin/dupr, however, I do not see this on the master branch. On which branch, or where else, can I find this?
>>>>>>>>> 
>>>>>>>> I was going to say, wait, it is there, and it has been there, hasn't it?
>>>>>>>> But I just looked at the install that I setup for the head of the master
>>>>>>>> from last Thurs., and yeah, I don't see that there.  Didn't Serge merge
>>>>>>>> that into master yet?
>>>>>>>> 
>>>>>>>> Douglas
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
> 

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1