 |
 |
Hi,Andy! Is it available via command line interface? artem. On Wed, 25 Oct 2006, Andrew Hanushevsky wrote: > Hi Pavel, > > Yes, look at XrdClientAdmin.hh. The XrdClientAdmin interface has a > Prepare() method that allows you to pass a list of files. This list is > passed to xrootd which then locates the files and locates them. If you > also specify kXR_stage then the files will be staged if not on disk. There > are additional options but for now this should be enough to get you going. > > That said, what we have found is that unless you have taken great care to > group files that will be likely used together on the same tape (something > we find difficult to do), the probability of improving tape access is > rather low unless you get a huge list of files into HPSS. That in itself > causes other problems. > > Andy > > On Wed, 25 Oct 2006, Pavel Jakl wrote: > > > Hi Andy and others, > > > > I would like to discuss the topic of having a possibility to "prepare" a > > list of files before user's jobs processing or at the beginning of the > > job. The fact why I am rising this topic is a optimization of the > > access to our tape system (HPSS). > > > > What we are doing now is that we organize the requests in a fashion that > > we have as many file requests as possible on the same tape at HPSS and > > therefore achieve better IO performance. Of course, one can imagine that > > when we would have bigger list than we can sort better and have more > > file's requests at the same tape. > > > > Our framework gives a possibility that user's lists within one job are > > already sorted in some fashion and with a big hope that those files are > > presumably at the same tape. The problem is that processing of these > > lists goes at subsequent order, one by one. > > So, my effort is to give to xrootd a list of files to "prepared" before > > processing them (or at least the full list would start to be prepared > > when job will start). Can I somehow "publish" to server that these files > > from the list, I will use in very close future ? > > > > So, I have figured 2 possible cases how to do it: > > > > 1) Use AsyncOpen at client side > > > > I am scared of this solution, since it can use lots of resources > > with many simultaneously opened connections. I can see jobs with > > thousand of files. In our case, when we have 400 nodes for job's > > processing, it could be very big number of connections to redirector node. > > > > 2) There is some sort of "prepare" methods at server side. So, how can I > > call them from the client side ? > > > > This solution would be better for me, if I could somehow pass the list > > to the server through the client and do not need any other assistance or > > presence of the client at each file's preparation. > > I know that there could be a problem with this. The files can disappear > > between the prepared and actual requested time of processing. (server > > went down, purging etc.). I think that the occurrence of this case is > > usually very small. However, I didn't make any sophisticated > > investigation to prove it since it would be very hard to get these > > statistic. > > > > Thanks for any suggestion or help > > Cheers > > Pavel > > >