On 3/28/19 5:31 PM, Michal Kamil Simon wrote:
> Hi Adrian,
Hi Michal!
> If (for now) you simply want to download a file having different
> replicas (with
> different names, etc.) a metalink is what you need. Have a look at:
cool! it is perfect for my need
> https://en.wikipedia.org/wiki/Metalink#Example_Metalink_4.0_.meta4_file
>
> you can create a metalink containing different URLs and assign priorities
> to them. You could e.g. generate the metalink from python and then use a
> local metalink as source in the CopyProcess.
>
> Once we release 4.9.1 you will be also able to do a multi-source download
> using a metalink.
>
> However, this wont work for uploads.
of course!
I made one for testing,
https://sev.spacescience.ro/adrian/meta/alice_cern.ch_user_a_asevcenc_testfile.meta4
but it seems that there are problems :
1. the url part is not parsed properly
2. the usage require a destination (but that is required and included in
the metalink) - so maybe if the source is meta4 the destination is taken
from the meta file and no longer required in cli?
I am sure that there is a problem with the url parsing because the same
url works for a normal add_job copy request
Additional question for when it will be working :
for the python part - add_Job, will take as source this .meta4 file?
Thank you!!
Adrian
>
> Cheers,
> Michal
> ________________________________________
> From: Adrian Sevcenco
> Sent: 28 March 2019 16:03
> To: Michal Kamil Simon; [log in to unmask]
> Subject: Re: xrootd python :: CopyProcess - stop the queue if a job succeed
>
> On 3/28/19 4:35 PM, Michal Kamil Simon wrote:
> > Hi Adrian,
> Hi!
>
> > My impression (correct me if I'm wrong) is that you are trying to do
> a multi-source transfer,
> > right?
> well, that would have been the optimal result, but at this moment i was
> just trying
> to download a file while trying all replicas
>
> > If so, you need to specify in the CopyProcess the sourceLimit, and as
> a source you need
> > to specify a metalink containing all the replicas, or a manager (in
> this case the client will
> > figure out what the replicas are by itself by using locate).
> my case is a little special ... the same file have multiple replicas
> that have _different_ physical names
> and authz envelopes ...
> as a client i have all required info.. i know for my logical file name
> /alice/cern.ch/user/a/asevcenc/testfile
> that the actual links are :
> root://eosalice.cern.ch:1094//06/17419/a7f1c690-4e49-11e9-8a64-0242266c4d21?_LONG_STRING_OF_OPTIONS_
> and
> root://ipngridxrd0.in2p3.fr:1094//06/17419/a7f1c690-4e49-11e9-8a64-0242266c4d21?_LONG_STRING_OF_OPTIONS_
>
> how can i create/what is the format of a metalink?
> can i somehow put toghether the remote links from above?
>
> would the same mechanism work for upload?
>
> > I just pushed a patch for the client that is enabling multi-source
> download in python bindings
> > (63e9604) and will port it for 4.9.1.
> great! thanks a lot!!!
>
> Adrian
>
>
> >
> > Cheers,
> > Michal
> > ________________________________________
> > From: [log in to unmask] [[log in to unmask]] on
> behalf of Adrian Sevcenco [[log in to unmask]]
> > Sent: 28 March 2019 14:22
> > To: [log in to unmask]
> > Subject: xrootd python :: CopyProcess - stop the queue if a job succeed
> >
> > Hi! In my use case (ALICE), to download a file, i get all replicas
> > and i add them as copy jobs to a CopyProcess ..
> > I would like to stop the CopyProcess when a job is succesful ..
> > Is there a way to do this?
> > Maybe in MyCopyProgressHandler.end() to check if results['status'] have
> > a "status: 0" and cancel all CopyProcess?
> > Is there a way to stop a CopyProcess? or other way to take into
> account the posibility
> > of multiple replicas?
> >
> > Thank you!
> > Adrian
> >
> >
> > ########################################################################
> > Use REPLY-ALL to reply to list
> >
> > To unsubscribe from the XROOTD-L list, click the following link:
> > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
> >
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
|