On 9/9/19 12:56 PM, Michal Kamil Simon wrote:
> Hi Adrian,
>
> Could you try adding the /xrdcl.unzip=AliAOD.root/ cgi to the original
> metalink URL, like:
>
> file://localhost/home/adrian/tmp/_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4?xrdcl.unzip=AliAOD.root
well, i do not think that the argument understand something else than a
file :
ll
total 3266976
-rw-r--r-- 1 adrian adrian 3345375231 Sep 9 13:06 AliAOD.root.zip
-rw-rw-r-- 1 adrian adrian 3289 Sep 9 13:02
_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4
xrdcp -p -P -f
_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4?xrdcl.unzip=AliAOD.root
AliAOD.root
xrdcp: No such file or directory processing
_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4?xrdcl.unzip=AliAOD.root
xrdcp -p -P -f
file://localhost/_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4?xrdcl.unzip=AliAOD.root
AliAOD.root
xrdcp: No such file or directory processing
/_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4?xrdcl.unzip=AliAOD.root
xrdcp -p -P -f
file://localhost/${PWD}/_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4?xrdcl.unzip=AliAOD.root
AliAOD.root
xrdcp: No such file or directory processing
//home/adrian/work-GRID/jalien_py/t/_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4?xrdcl.unzip=AliAOD.root
xrdcp -p -P -f
file:///localhost/${PWD}/_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4?xrdcl.unzip=AliAOD.root
AliAOD.root
xrdcp: No such file or directory processing
/localhost//home/adrian/work-GRID/jalien_py/t/_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4?xrdcl.unzip=AliAOD.root
xrdcp -p -P -f
file://${PWD}/_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4?xrdcl.unzip=AliAOD.root
AliAOD.root
xrdcp: No such file or directory processing
/home/adrian/work-GRID/jalien_py/t/_alice_data_2018_LHC18m_000291397_pass1_withTRDtracking_AOD208_0884_AliAOD.root.meta4?xrdcl.unzip=AliAOD.root
Thank you!
Adrian
>
> (it seems that there's a friction between the metalink handling and zip
> archive handling)
>
> Cheers,
> Michal
> ________________________________________
> From: Adrian Sevcenco
> Sent: 06 September 2019 16:30
> To: Michal Kamil Simon; [log in to unmask]
> Subject: Re: python :: cp process fails when it shouldn't (another
> utility can download file)
>
> Hi Michal!
>
> On 9/2/19 5:52 PM, Michal Kamil Simon wrote:
> > Yep, it looks about right :-)
> so, i use ?xrdcl.unzip=AliAOD.root&authz=TOKEN
> format but it seems that the zipfile is downloaded instead of file
> extracted from archive...
>
> the detailed log is here:
> https://cernbox.cern.ch/index.php/s/JNaLKsaC5pyrhMP
>
> do you have any idea/hint why the file is not extracted?
>
> Thanks a lot!!
> Adrian
>
>
>
> >
> > Michal
> > ________________________________________
> > From: Adrian Sevcenco
> > Sent: 02 September 2019 16:46
> > To: Michal Kamil Simon; [log in to unmask]
> > Subject: Re: python :: cp process fails when it shouldn't (another
> utility can download file)
> >
> > On 9/2/19 5:29 PM, Michal Kamil Simon wrote:
> >> Hi Adrian,
> > Hi!
> >
> >> You can replace the '#' with '?xrdcl.unzip=' however you have to make
> >> sure that if the URL
> >> already contains a CGI you have to replace the following '?' with '&',
> >> e.g. :
> >>
> >>
> root://eosalice.cern.ch:1094//15/62933/56da6906-9149-11e7-ba1b-579516ed5c66*#*AliAOD.root*?*authz=LONG_ALICE_TOKEN
> >>
> >> gets transformed into:
> >>
> >>
> root://eosalice.cern.ch:1094//15/62933/56da6906-9149-11e7-ba1b-579516ed5c66*?xrdcl.unzip=*AliAOD.root*&*authz=LONG_ALICE_TOKEN
> >
> > yeap, is doable as the full url (physical url + token) is constructed by
> > me... so i can do something like :
> >
> > pfn_components = pfn.split('#') # i have the guarantee that the files
> > that ALICE uploads have no # in name
> >
> > if len(pfn_components) > 1:
> > full_url = pfn + '?xrdcl.unzip=' + pfn_components[1] +
> > '&authz=LONG_ALICE_TOKEN'
> > else:
> > full_url = pfn + '?authz=LONG_ALICE_TOKEN'
> >
> > Does it sound right?
> > Thanks a lot for help!!
> > Adrian
> >
> >
> >
> >>
> >>
> >>
> >> Regarding the /CopyProcess.add_job(...)/ method I could add parameters
> >> that will allow
> >> to specify the file name for extraction from zip archive.
> >>
> >> Regarding supporting the '#' root native format we will have to see with
> >> Andy whether this
> >> wont harm any existing use cases (as # is a legal character that could
> >> be used a file name).
> >>
> >> Cheers,
> >> Michal
> >> ________________________________________
> >> From: Adrian Sevcenco
> >> Sent: 02 September 2019 15:37
> >> To: Michal Kamil Simon; [log in to unmask]
> >> Subject: Re: python :: cp process fails when it shouldn't (another
> >> utility can download file)
> >>
> >> On 9/2/19 2:09 PM, Michal Kamil Simon wrote:
> >> > Hi Adrian,
> >> Hi!
> >>
> >> > >From what I see in the logs you use the following file name:
> >> >
> >> >
> >>
> root://eosalice.cern.ch:1094//15/62933/56da6906-9149-11e7-ba1b-579516ed5c66#AliAOD.root
> >> >
> >> > The '#' is root syntax for unpacking root files, this is not supported
> >> > in the
> >> > xrootd client, instead you have to use the /xrdcl.unzip/ cgi tag, e.g.
> >> >
> >> >
> >>
> root://eosalice.cern.ch:1094//15/62933/56da6906-9149-11e7-ba1b-579516ed5c66?xrdcl.unzip=AliAOD.root
> >>
> >> oh!!! so, could i use a simplistic logic like :
> >> replace latest '#' from string with '?xrdcl.unzip='
> >>
> >> ALICE stores files in the form of GUID (that last uid)
> >> and when i request access to a lfn i get the guid and the authz envelope
> >> for accessing the file ... so, it is guaranteed that i will always get a
> >> url with a GUID ...
> >>
> >> Given this, do you thing that i could use the logic from above?
> >>
> >> > alternatively I can expose extracting of zip files (root files use zip
> >> > format for bundling)
> >> > in the /CopyProcess.add_job(...)/ method.
> >> that would be great! if it is possible it would be best if
> >> the same format of '#file' is recognized (as this is the url that i get
> >> when requesting lfn access)
> >>
> >> Thanks a lot!!
> >> Adrian
> >>
> >> >
> >> > Hope this helps!
> >> >
> >> > Cheers,
> >> > Michal
> >> >
> >> > ________________________________________
> >> > From: Adrian Sevcenco
> >> > Sent: 01 September 2019 22:13
> >> > To: [log in to unmask]
> >> > Cc: Michal Kamil Simon
> >> > Subject: python :: cp process fails when it shouldn't (another utility
> >> > can download file)
> >> >
> >> > Hi! I have a really baffling situation where my python tool cannot
> >> > download a file and another tool (java based, use xrdcp) can download
> >> > the same file ...
> >> >
> >> > the detailed logs for my cp are here :
> >> > https://cernbox.cern.ch/index.php/s/JNaLKsaC5pyrhMP
> >> >
> >> > the java based tool it seems that somehow ignores the external XRD_
> >> > variables so i cannot get a log of cp process
> >> >
> >> > Could some expert take a look please and point me to a hint why my cp
> >> > fails and the other tool can download just fine?
> >> >
> >> > Thank you!!
> >> > Adrian
> >> >
> >>
> >>
> >> --
> >> ----------------------------------------------
> >> Adrian Sevcenco, Ph.D. |
> >> Institute of Space Science - ISS, Romania |
> >> adrian.sevcenco at {cern.ch,spacescience.ro} |
> >> ----------------------------------------------
> >>
> >
> >
> > --
> > ----------------------------------------------
> > Adrian Sevcenco, Ph.D. |
> > Institute of Space Science - ISS, Romania |
> > adrian.sevcenco at {cern.ch,spacescience.ro} |
> > ----------------------------------------------
> >
> >
> >
>
>
> --
> ----------------------------------------------
> Adrian Sevcenco, Ph.D. |
> Institute of Space Science - ISS, Romania |
> adrian.sevcenco at {cern.ch,spacescience.ro} |
> ----------------------------------------------
>
--
----------------------------------------------
Adrian Sevcenco, Ph.D. |
Institute of Space Science - ISS, Romania |
adrian.sevcenco at {cern.ch,spacescience.ro} |
----------------------------------------------
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
