Ashiq,
In STAR, the STAR Scheduler knows about the datasets
and their locations and scheduling through SUMS coupled with
Xrootd data access makes what you want to do possible. In
other words
- we know that data is at place A and B, can chose between
A or B (exclusive or not)
- if the file end up no being there, Xrootd data access with
MSS plugin will restore the file for you
- Later on, SRM behind (not in front) will allow moving data
to long distances ... but this is possible now with Xrootd
too
In other words, minimal data movement is made but you
CANNOT avoid defining your broker algorithm. The questions are
then things like "is it better to run on the other side of the
country or the planet because the data is there ... or wait
until I get a slot close-by". All factored in (network perf,
CPU, queue length) this is not easy but possible in STAR (we
can swap algorithm or "policies", test new ones etc ...).
Is this what you need ??
Note: without Xrootd, the problem is MUCH more complicated.
In its native way, Xrootd do not schedule jobs. I think this is
not necessary at this stage (sounds to me that the IO performance
enhancement and capability enhancement are more important).
Ashiq Anjum wrote:
> I want to do data intensive Grid scheduling. For example, I plan to
> schedule a compute job that will do certain operations on a dataset.
> Should data be moved to job or job should be moved towards data or
> any other related decision making.
>
> My query is, can xrootd in any way become such a scheduling platform
> or it will remain only a data access mechanism in which scheduler
> will request xrootd to access cetain datasets?
>
> cheers, Ashiq Anjum
>
>
> -----Original Message----- From: Jerome LAURET
> [mailto:[log in to unmask]] Sent: Mon 6/12/2006 9:21 PM To: Ashiq Anjum
> Cc: Peter Elmer; [log in to unmask] Subject: Re: Scheduling
> with xrootd
>
>
> What does "grid scheduling with Xrootd" means ??? You need to expand
> really. There are Meta-Schedulers aimed to do diverse task (including
> scheduling where the data is if knowledge is available). Xrootd could
> "place" the data bringing it from a distance or MSS.
>
> What are you after ??
>
> Ashiq Anjum wrote:
>> Hi Peter,
>>
>> Thanks for your quickly reply.
>>
>> Actually I want to know can i do Grid Scheduling with xrootd? Is
>> that possible with PROOF?
>>
>> cheers,
>>
>> Ashiq Anjum
>>
>>
>>
>> -----Original Message----- From: Peter Elmer
>> [mailto:[log in to unmask]] Sent: Mon 6/12/2006 9:01 PM To: Ashiq
>> Anjum Cc: [log in to unmask] Subject: Re: Scheduling with
>> xrootd
>>
>> Hi Ashiq,
>>
>> On Mon, Jun 12, 2006 at 09:55:37PM +0200, Ashiq Anjum wrote:
>>> I have a quick question. Can we do
>>> Meta-Scheduling/co-scheduling(scheduling of compute jobs taking
>>> into account data location) with xrrotd? What I found that xrootd
>>> is only a data access mechanism in a decentralized way and I will
>>> be grateful if I can be proven wrong:)
>> As designed it is a data access mechanism.
>>
>> Are you asking for PROOF or for something else?
>>
>> Pete
>>
>> -------------------------------------------------------------------------
>> Peter Elmer E-mail: [log in to unmask] Phone: +41 (22)
>> 767-4644 Address: CERN Division PPE, Bat. 32 2C-14, CH-1211 Geneva
>> 23, Switzerland
>> -------------------------------------------------------------------------
>>
>>
>
--
,,,,,
( o o )
--m---U---m--
Jerome
