 |
 |
Hi,Fabrizio! thanks, but sorry - what's the tool? :( Artem. On Thu, 26 Oct 2006, Fabrizio Furano wrote: > Hi Artem, > > yes, it is. For other commands just type "help" at the command prompt. > If you have any problems, let me know. > > Fabrizio > > Artem Trunov wrote: > > Hi,Andy! > > > > Is it available via command line interface? > > > > artem. > > > > On Wed, 25 Oct 2006, Andrew Hanushevsky wrote: > > > >> Hi Pavel, > >> > >> Yes, look at XrdClientAdmin.hh. The XrdClientAdmin interface has a > >> Prepare() method that allows you to pass a list of files. This list is > >> passed to xrootd which then locates the files and locates them. If you > >> also specify kXR_stage then the files will be staged if not on disk. There > >> are additional options but for now this should be enough to get you going. > >> > >> That said, what we have found is that unless you have taken great care to > >> group files that will be likely used together on the same tape (something > >> we find difficult to do), the probability of improving tape access is > >> rather low unless you get a huge list of files into HPSS. That in itself > >> causes other problems. > >> > >> Andy > >> > >> On Wed, 25 Oct 2006, Pavel Jakl wrote: > >> > >>> Hi Andy and others, > >>> > >>> I would like to discuss the topic of having a possibility to "prepare" a > >>> list of files before user's jobs processing or at the beginning of the > >>> job. The fact why I am rising this topic is a optimization of the > >>> access to our tape system (HPSS). > >>> > >>> What we are doing now is that we organize the requests in a fashion that > >>> we have as many file requests as possible on the same tape at HPSS and > >>> therefore achieve better IO performance. Of course, one can imagine that > >>> when we would have bigger list than we can sort better and have more > >>> file's requests at the same tape. > >>> > >>> Our framework gives a possibility that user's lists within one job are > >>> already sorted in some fashion and with a big hope that those files are > >>> presumably at the same tape. The problem is that processing of these > >>> lists goes at subsequent order, one by one. > >>> So, my effort is to give to xrootd a list of files to "prepared" before > >>> processing them (or at least the full list would start to be prepared > >>> when job will start). Can I somehow "publish" to server that these files > >>> from the list, I will use in very close future ? > >>> > >>> So, I have figured 2 possible cases how to do it: > >>> > >>> 1) Use AsyncOpen at client side > >>> > >>> I am scared of this solution, since it can use lots of resources > >>> with many simultaneously opened connections. I can see jobs with > >>> thousand of files. In our case, when we have 400 nodes for job's > >>> processing, it could be very big number of connections to redirector node. > >>> > >>> 2) There is some sort of "prepare" methods at server side. So, how can I > >>> call them from the client side ? > >>> > >>> This solution would be better for me, if I could somehow pass the list > >>> to the server through the client and do not need any other assistance or > >>> presence of the client at each file's preparation. > >>> I know that there could be a problem with this. The files can disappear > >>> between the prepared and actual requested time of processing. (server > >>> went down, purging etc.). I think that the occurrence of this case is > >>> usually very small. However, I didn't make any sophisticated > >>> investigation to prove it since it would be very hard to get these > >>> statistic. > >>> > >>> Thanks for any suggestion or help > >>> Cheers > >>> Pavel > >>> >