XROOTD-L Archives

Support use of xrootd by HEP experiments

XROOTD-L@LISTSERV.SLAC.STANFORD.EDU

Options: Use Forum View

Use Monospaced Font
Show Text Part by Default
Show All Mail Headers

Message: [<< First] [< Prev] [Next >] [Last >>]
Topic: [<< First] [< Prev] [Next >] [Last >>]
Author: [<< First] [< Prev] [Next >] [Last >>]

Print Reply
Subject:
From:
Sam Skipsey <[log in to unmask]>
Reply To:
Support use of xrootd by HEP experiments <[log in to unmask]>
Date:
Thu, 14 Nov 2019 20:39:00 +0000
Content-Type:
text/plain
Parts/Attachments:
text/plain (154 lines)
On Thu, 14 Nov 2019 at 20:27, Matevž Tadel <[log in to unmask]> wrote:
>
> Hi,
>
> You are missing oss.localroot ... this has to be always defined. That's
> where symlinks to various oss.space instances go.
>

...how does this work (and why isn't it described in any detail how
this applies to oss.spaces in the documentation?)

> Looking at your config, you don't seem to need oss.space and pfc.spaces
> at all, you just need oss.localroot /storage0/
>

No, we have /storage0 through /storage4.

> It seems the disks are aggregated in some other ways, not by being added
> to any space.
>

That's... partly true. But the /storage* filesystem locations are all
distinct, and associated with different devices.

> public is the default space used by xcache, for both data and metadata
> (so what you specify are actually the defaults).
>

I know, but since it wasn't working, I thought I'd put it in
explicitly just in case.

> xcache allows you to separate spaces for data and metadata, often it is
> useful to have metadata and localroot on ssd, or at least on a separate
> disk / controller.
>

Indeed... as noted, the problem is that there's no documentation about
how to associate a space with the localroot, as the documentation
doesn't talk about spaces in this context.

> See the relevant lines from a config we use at UCSD below.
>
> It turns out that for cache it is better to use raw disks than raid5/6
> or zfs as you get independent streams for each file being opened going
> to specific disks, both on input and output. Data loss due to a lost
> disk is not a problem for caches.
>
> I hope this is clearer now :)
>

Not really: I still need to know how you associate the localroot with a space.

> Matevz
>
> oss.localroot /xcache-root  # On SSD
>
> oss.space data /data1/xcache # Spinning disks (we use xfs)
> oss.space data /data2/xcache
> oss.space data /data3/xcache
> oss.space data /data4/xcache
> oss.space data /data5/xcache
> oss.space data /data6/xcache
> oss.space data /data7/xcache
> oss.space data /data8/xcache
> oss.space data /data9/xcache
> oss.space data /data10/xcache
> oss.space data /data11/xcache
> oss.space data /data12/xcache
>
> oss.space meta /xcache-meta # On SSD
>
> # Use space "data" for data, "meta" for meta-data/cinfo files
> pfc.spaces data meta

Yes. So... is /xcache-root a real path? Is /xcache-meta a real path?

This doesn't help me with what I should point oss.localroot at.

Sam

>
> On 2019-11-14 11:50, Sam Skipsey wrote:
> > Hello,
> > So, I've tried various interpretations of the documentation, but I'm
> > obviously missing something.
> >
> > I'm trying to configure an Xrootd Proxy Cache, backed by storage
> > aggregated into an oss.space.
> >
> > The cache is configured like:
> >
> >
> > oss.space public /storage0*
> > #oss.space public default /
> >
> > ofs.osslib libXrdPss.so
> > pss.cachelib libXrdFileCache.so
> > pfc.ram 16g
> > all.trace all
> > all.log all
> > pfs.diskusage 0.90 0.95
> >
> > all.export /xroot:/ outplace
> > all.export /root:/ outplace
> >
> > all.export / outplace
> >
> > pss.origin svr01.beowulf.cluster:1094
> > xrd.allow host *.beowulf.cluster
> >
> > pfc.blocksize 256k
> >
> > pfc.spaces public public
> >
> >
> >
> > where the /storage0* spaces are all writable and owned by the xrootd
> > user (and when we start up the server, it creates "public" directories
> > in each of them).
> >
> > However, whenever I try to access a file through the proxy, it logs
> > that it couldn't write the file to the disk (specifying its path as if
> > it was going to write it to the storage with no prefix - that is,
> > outside of the specified "public" space, in the literal root
> > filesystem).
> > If this was a non-spaces based system, I'd specify oss.localroot, but
> > that doesn't seem to apply to a space?
> > I tried adding the outplace directives to the all.exports, but this
> > hasn't changed anything.
> >
> > What am I missing? None of the example configurations I can see with
> > oss.spaces seem to need any additional configuration directives...
> >
> > Sam
> >
> > ########################################################################
> > Use REPLY-ALL to reply to list
> >
> > To unsubscribe from the XROOTD-L list, click the following link:
> > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
> >
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1

ATOM RSS1 RSS2