Let's think about qserv and tapes and discuss tomorrow
during our hangout


-------- Forwarded Message --------
Subject: Re: Querying databases on tape
Date: Tue, 9 Feb 2016 09:29:08 -0800
From: Jacek Becla <[log in to unmask]>
Organization: SLAC National Accelerator Laboratory
To: Mario Juric <[log in to unmask]>

That is why I think we can do it for interactive queries,
we would fetch just the chunk / few chunks needed. I'll brainstorm
with my folks tomorrow about how difficult it might be to implement
fetching from tape. But I suspect by the time we need it, there
will be some sort of disk-like low-power technology better than
tapes for keeping everything online

Jacek


On 02/09/2016 09:21 AM, Mario Juric wrote:
>
>> On Feb 9, 2016, at 8:59 , Jacek Becla <[log in to unmask]> wrote:
>>
>> Hi Mario
>>
>> We haven't worked out the details but in principal it should
>> be possible for xrootd to fetch data from tape and make it
>> available for local mysql on the worker. xrootd already can
>> do it, so it is a question of writing the glue code in qserv layer.
>>
>
> Yes, but it’s a question of capacity, isn’t it? If you’re in DR7, and you have one outstanding query for DR1,2,3,4,5, will you fetch them from tape in parallel (thus requiring more disk and IO) or not?
>
>> Note that this will not be needed until some 2023-2024,
>> I wasn't going to worry about it any time soon.
>>
>
> OK, understood — I mostly need to make sure I know what is the current baseline; we (Jeff, K-T, and I) have been telling folks outside of the project that the older DRs will be accessible only as bulk downloads. That’s not bad re managing expectations, but that’s where a lot of the pressure to keep everything on disk has been coming from. If we didn’t get that right, I need to get us on the right wavelength.
>
> Cheers,
> - M.
>
>> Jacek
>>
>>
>> On 02/09/2016 08:42 AM, Mario Juric wrote:
>>> Hi Jacek,
>>> 	Just to check — re https://jira.lsstcorp.org/browse/RFC-134?focusedCommentId=44095&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-44095 — are you saying that even with the current baseline, you’re building the capability to field queries to DRx-2 databases that would be fetched from tape, instantiated into Qserv, run, and then wiped away?
>>>
>>> Cheers,
>>> —
>>> Mario Juric,
>>> UW Astronomy Faculty | UW eScience | LSST DM Project Scientist
>>> Web : http://research.majuric.org      Phone : +1 609 933 1033
>>>

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the QSERV-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=QSERV-L&A=1