 |
 |
On Oct 19, 2006, at 10:09 AM, Pavel Jakl wrote: > Hi guys, > > I am not sure that this is what Dan is trying to find. He would > like to integrate his IBP into xrootd, not xrootd into IBP. What I > understand from his email is to have a yes IBP into xrootd > IBP as a underlying file system. I am not quite sure, why he is > trying to do it ? If he would like to take an advantage of the load > balancing, aggregation or something else, what is done in xrootd in > very efficient way. > Dan, can you tell us what are your expectations from it ? > IBP with L-Store promises convenient and efficient access to large amounts of distributed storage: L-Store/IBP provides the option for replacing local disks with efficient networking. if interested check out: http://www.lstore.org/mwiki/index.php/ Main_Page we are looking at hosting CMS data on distributed IBP depots and we would like to provide a service so root users can stream data directly from IBP into an interactive root session. ibp load balancing or similar network-usage optimization will eventuallly be handled by the l-store layer, so xrootd+ibp+lors/lstore is only being used (initially) as a point to gather ibp data and serve it via protocols root can understand. the other SRM work may also be a separate pathway for connecting root to ibp and we'll look into other things as well my understanding is that currently these SRM interface projects are not yet ready for testing, or are just becoming available. thanks, Dan > There was a recent work of integrating Berkeley SRM (at this time, > just HRM) as a backend storage and I think, this what he would like > to see. Client has posix functions, but server not or I am not > sure about them. > You can take a look at Andy's recent presentation at CERN of > integrating SRM into xrootd. There are slides explaining what you > need to do, what to extend etc. > > http://indico.cern.ch/materialDisplay.py? > contribId=s0t1&sessionId=s0&materialId=0&confId=a062357 > > In other words, you would like to extend server's side oss layer > and instead of file system calls, call your IBP io commands. > You cannot see the Berkeley SRM work on the latest version of > server which is available at the website. > > As I said, I am not sure that this is what you are trying to do. > So, maybe, I am totally out of topic :-) > > Cheers > Pavel > > > Fabrizio Furano wrote: >> Hi, >> >> what JY says is correct, the only thing I want to add is that, >> since you are writing C/C++ code (it seems) you may directly use >> xrdclient, avoiding the posix wrappers which are built on it. Just >> have a look at XrdClient.hh for the public interface. >> >> Fabrizio >> >> >> Jean-Yves Nief wrote: >>> hello Dan, >>> >>> Daniel Engh wrote: >>>> Hi, >>>> >>>> I am from Vanderbilt University trying to hook up the Vanderbilt/ >>>> UT-Knoxville L-Store/IBP storage backend to xrootd. >>>> >>>> basically IBP is a bit-torrent like distributed data-storage >>>> system, LORS adds some file-system functionality analogous to >>>> inodes, and the new Vanderbilt L-Store is an SRM complient meta- >>>> data layer on top of IBP. >>>> >>>> I have been playing with the latest development version of >>>> xrootd downloadable from the web: >>>> xrootd-20060523-1741.src.tgz >>>> >>>> I initially want to do a very simple file streaming test of a >>>> root file on ibp. I'm not an expert on posix i/o but I think I >>>> need only >>>> use my posix-complient methods: >>>> int fd = xio_open(my_uri, O_RDONLY); >>>> and >>>> xio_read(fd, buffer, bufsize) >>>> >>>> I need then 2 things from xrootd: >>>> 1. location of the posix i/o calls I should modify >>>> ie. where is an xrootd posix open() call I can replace with >>>> xio_open() >>>> ie. and then and xrootd posix read() call I can replace with >>>> xio_read() >>>> 2. how do I launch xrootd to use my modified xrootd module >>>> 3. how do I use root to call this xrootd/ibp streamed file. >>>> >>>> I've tested my lors/ibp libs with a simple c++ Hello IBP-world >>>> program. >>>> And I can compile IBP libs into xrootd and get things like an >>>> xrootd init() call >>>> to call xio_open() xio_read() and print out a hello-ibp message. >>> on the client side, the Xrootd POSIX functions are for example: >>> XrdPosix_Open, XrdPosix_Lseek, XrdPosix_Read etc... >>> don't forget to include: >>> XrdPosix/XrdPosix.hh >>> you might take a look at the XrdPosix package in any case >>> (README, etc...). You will find instructions on how to build the >>> file URL in order to open it through xrootd, compile etc... If >>> you wish, I can send you some sample code to use the posix >>> clients + the make file. >>> one other possibility (providing you are using the regular unix >>> calls: open, read, write, but that is not your case at the >>> moment) is to use the preload library. In that case, you can talk >>> to a xrootd server without changing any single line of code (and >>> without recompiling): the unix calls are redirected to the >>> xrootd calls. It is a pretty nice feature, we have an experiment >>> which is using it here (it has some little disadvantages). >>> hope this help, >>> JY >>>> >>>> Now I just need the correct place in xrootd to put in my ibp io >>>> commands. >>>> >>>> Eventually I'd like to make xrootd recognize a couple of >>>> protocols named: >>>> lors:// and lstore:// but for now I can masquerade as some >>>> other protocol >>>> and just set up an xrootd service that is hard-coded to only >>>> look into its IBP setup. >>>> >>>> thanks for any suggestions on how to get the latest code and >>>> trying some things out, >>>> Dan >>>> >