 |
 |
Hello Sam Could you check your config file and use cms.allow ..... instead of cmsd.allow. Otherwise your configuration should be fine. The xrootd should put your file into the /gridstore file systems and there should be a link to these files in /disk/... Can you write files directly to the data server? for example is xrdcp somefile root://disk042.gla.scotgrid.ac.uk//disk/file1 working? Are you using the xrootd port 2525? If so your xrdcp command should look like: xrdcp somefile root://disk042.gla.scotgrid.ac.uk:2525//disk/file1 How big are your file systems and free space? By default the cmsd requires that the free space is 10GB or 2% what ever is larger. And if there is no server with sufficient free space you will not be able to write files through the redirector. You could also turn on debugging: cms.trace all and send me the output and I could have a look at it. Cheers, Wilko On Thu, 7 May 2009, Sam Skipsey wrote: > Hi Wilko, > > That's great - your version of my configuration works immediately > without the read-only problem I was having (and no other changes > needed.) The only tweak I needed was adding > xrootd.allow host disk042.gla.scotgrid.ac.uk > cmsd.allow host disk042.gla.scotgrid.ac.uk > > So, now I'm trying something a little more advanced: > I replaced the all.export with > > oss.cache public /gridstore0 xa > oss.cache public /gridstore1 xa > oss.cache public /gridstore2 xa > > all.export /disk r/w > > and added an > xrootd.allow host disk*.gla.scotgrid.ac.uk > cmsd.allow host disk*.gla.scotgrid.ac.uk > > in place of the specific disk042 > > I then stuck identically configured xrootd + cmsd services on other > disk servers in this range, and could see them connecting to svr025 > after bringing it's services back up. > > I think that this should allow me to do > > xrdcp somefile root://svr025.gla.scotgrid.ac.uk//disk/file1 > > and have the file1 actually be stored on one of the gridstore filesystems. > What actually happens is that I get: > > No servers are available to write the file. > > Which looks like something is confused somewhere? > Do I need to explictly tell xroot that it needs to use the cache as > the storage location for /disk related transfers? > > Sam > > 2009/5/6 Wilko Kroeger <[log in to unmask]>: >> >> Hello Sam >> >> Just a few comments. >> As you found out the example config files on the slac web page are quite old >> and out dated. I will take them down and replace them with a newer version. >> The configuration options changed a bit they are easier now. >> Also if possible you should use the cmsd instead of the olbd, >> I modified your config files and made one out of it that will work with both >> your data server and redirector. >> >> # config file for both dataserver and redirector >> >> all.manager svr025.gla.scotgrid.ac.uk 2526 >> all.export /gridstore4 r/w >> if svr025.gla.scotgrid.ac.uk >> all.role manager >> else >> all.role server >> fi >> >> xrootd.fslib /root/20090202-1402/lib/libXrdOfs.so >> xrootd.allow host svr025.gla.scotgrid.ac.uk >> >> xrd.protocol xrootd * >> xrd.port 2525 >> >> cms.allow host svr025.gla.scotgrid.ac.uk >> cms.trace redirect >> >> >> In order to start the xrootd and cms you can run: >> /root/20090202-1402/bin/xrootd -c <configfile> -l <logfile> >> /root/20090202-1402/bin/cmsd -c <configfile> -l <logfile> >> >> If there is already a xrootd running on one of these machine you have to use >> the name option (-n) for the xrootd/cmsd, e.g.: >> /root/20090202-1402/bin/xrootd -n mytest -c <configfile> -l <logfile> >> /root/20090202-1402/bin/cmsd -n mytest -c <configfile> -l <logfile> >> >> You could also use the StartXRD, StartCMS scripts which are in the etc dir >> of the xrootd release but for this you have to configure the StartXRD.cf >> script. >> >> I hope this helps. Please let me know if you have more questions. >> >> Cheers, >> Wilko >> >> >> On Wed, 6 May 2009, Fabrizio Furano wrote: >> >>> Hi Sam, >>> >>> so, if your preference is for the cern all-in-one bundle, then all the >>> instructions are in the cern url which I sent to you. >>> >>> Changing the ports is trivial and does not require low level tweaking, >>> since the values are in the meta-configuration file system.cnf. Just put >>> reasonable values following the criteria which are there. >>> >>> My advice is to setup/run tests in a machine which is not in production, >>> even a linux laptop is 100% ok. Just for convenience and ease of operation. >>> >>> Just a question: what is your test setup supposed to be for? >>> >>> Fabrizio >>> >>> Sam Skipsey wrote: >>>> >>>> Hi Fabrizio, >>>> >>>> So, I tried setting oss.localroot (and r/w) , but the data server >>>> still insists that it doesn't have write access to the filesystem and >>>> so sets itself to read only. >>>> Since it is being run as dpmmgr user, and the filesystem is owned by >>>> that user and is mode 0700, I'm at a bit of a loss to explain this. >>>> >>>> Thank you for the CERN link - I've been using the packages from the >>>> slac link, so at least the version of xrootd I have is reasonably up >>>> to date. Unfortunately, I doubt that I can directly use the all-in-one >>>> version without modifying it anyway, since I need to run the services >>>> on non-standard ports - I'm trying to run this initial test xrootd >>>> service on a head node which already hosts a dpm server (and >>>> dpm-xrootd plugins), so I can't use the default ports. >>>> (The style of the configuration files is based on the Example files >>>> provided on the SLAC website. Perhaps the examples need a bit of >>>> updating ;) ) >>>> >>>> Sam >>>> >>>> 2009/5/6 Fabrizio Furano <[log in to unmask]>: >>>>> >>>>> Hi Sam, >>>>> >>>>> you should specify r/w in the export directive of the data server, >>>>> writing >>>>> this: >>>>> >>>>> xrootd.export /gridstore4 r/w >>>>> >>>>> Then please verify that you are specifying the correct localroot >>>>> option. >>>>> >>>>> Moreover, the config files you wrote look really obsolete, and probably >>>>> valid for very old versions. My suggestion is to get a newer package and >>>>> refer to the online documentation to configure it. Things are much >>>>> simpler >>>>> now, in the sense that fewer directives are needed. >>>>> >>>>> Depending on your preferences, there are two flavors of the bundle >>>>> which >>>>> you can get: the vanilla one and the 'automatic all-in-one' one. >>>>> Respectively, you can find the first here: >>>>> >>>>> http://xrootd.slac.stanford.edu >>>>> (or you can take the tarball from the last ROOT version) >>>>> >>>>> and the second here: >>>>> >>>>> http://savannah.cern.ch/projects/xrootd >>>>> >>>>> Fabrizio >>>>> >>>>> >>>>> Sam Skipsey wrote: >>>>>> >>>>>> Hello, all: >>>>>> >>>>>> I may be being stupid here, but: >>>>>> >>>>>> I'm in the initial phase of messing around with xrootd here at >>>>>> Glasgow, and I can't seem to get my diskserver to add itself as >>>>>> writeable. >>>>>> >>>>>> The arrangement is: >>>>>> svr025.gla.scotgrid.ac.uk running as a redirection server (xrootd on >>>>>> port 2525 and olbd on 2526) >>>>>> disk042.gla.scotgrid.ac.uk (for example) running as a data server >>>>>> (same services on same ports) >>>>>> >>>>>> The config for the two is: >>>>>> >>>>>> redirector: >>>>>> # >>>>>> # redirectserver.cf >>>>>> # >>>>>> # xrootd >>>>>> xrootd.port 2525 >>>>>> xrootd.fslib /root/20090202-1402/lib/libXrdOfs.so >>>>>> xrootd.export /gridstore4 >>>>>> #oss.readonly >>>>>> odc.manager svr025.gla.scotgrid.ac.uk 2526 >>>>>> odc.trace redirect >>>>>> # olbd >>>>>> olb.port 2526 >>>>>> olb.allow host disk042.gla.scotgrid.ac.uk >>>>>> olb.allow host svr025.gla.scotgrid.ac.uk >>>>>> ofs.redirect remote svr025.gla.scotgrid.ac.uk >>>>>> xrootd.allow host svr025.gla.scotgrid.ac.uk >>>>>> >>>>>> >>>>>> dataserver: >>>>>> # >>>>>> # dataserver.cf >>>>>> # >>>>>> # xrootd >>>>>> xrootd.port 2525 >>>>>> xrootd.fslib /root/20090202-1402/lib/libXrdOfs.so >>>>>> xrootd.export /gridstore4 >>>>>> #oss.readonly >>>>>> odc.manager svr025.gla.scotgrid.ac.uk 2526 >>>>>> # olbd >>>>>> olb.port 2526 >>>>>> olb.subscribe svr025.gla.scotgrid.ac.uk 2526 >>>>>> ofs.redirect remote svr025.gla.scotgrid.ac.uk >>>>>> oft.redirect target >>>>>> >>>>>> >>>>>> However, when disk042 connects to the redirection server, I see: >>>>>> >>>>>> 090506 14:19:21 31690 olb_Manager: server disk042 defaulted r / >>>>>> >>>>>> and on the disk: >>>>>> 090506 14:18:01 001 olb_Meter: Warning! No writable filesystems found; >>>>>> write access and staging prohibited. >>>>>> >>>>>> and, indeed, xrdcp to disk042 fails (but copying from it works >>>>>> perfectly). >>>>>> The exported filesystem (/gridstore4) is writable by the user owning >>>>>> the xrootd processes, however (indeed, it is owned by the same user, >>>>>> with mode 770). >>>>>> >>>>>> This is probably something stupid, but... any help appreciated. >>>>>> >>>>>> Sam Skipsey >>>>>> Glasgow >>> >> >