 |
 |
On Thu, 14 Nov 2019 at 20:27, Matevž Tadel <[log in to unmask]> wrote: > > Hi, > > You are missing oss.localroot ... this has to be always defined. That's > where symlinks to various oss.space instances go. > ...how does this work (and why isn't it described in any detail how this applies to oss.spaces in the documentation?) > Looking at your config, you don't seem to need oss.space and pfc.spaces > at all, you just need oss.localroot /storage0/ > No, we have /storage0 through /storage4. > It seems the disks are aggregated in some other ways, not by being added > to any space. > That's... partly true. But the /storage* filesystem locations are all distinct, and associated with different devices. > public is the default space used by xcache, for both data and metadata > (so what you specify are actually the defaults). > I know, but since it wasn't working, I thought I'd put it in explicitly just in case. > xcache allows you to separate spaces for data and metadata, often it is > useful to have metadata and localroot on ssd, or at least on a separate > disk / controller. > Indeed... as noted, the problem is that there's no documentation about how to associate a space with the localroot, as the documentation doesn't talk about spaces in this context. > See the relevant lines from a config we use at UCSD below. > > It turns out that for cache it is better to use raw disks than raid5/6 > or zfs as you get independent streams for each file being opened going > to specific disks, both on input and output. Data loss due to a lost > disk is not a problem for caches. > > I hope this is clearer now :) > Not really: I still need to know how you associate the localroot with a space. > Matevz > > oss.localroot /xcache-root # On SSD > > oss.space data /data1/xcache # Spinning disks (we use xfs) > oss.space data /data2/xcache > oss.space data /data3/xcache > oss.space data /data4/xcache > oss.space data /data5/xcache > oss.space data /data6/xcache > oss.space data /data7/xcache > oss.space data /data8/xcache > oss.space data /data9/xcache > oss.space data /data10/xcache > oss.space data /data11/xcache > oss.space data /data12/xcache > > oss.space meta /xcache-meta # On SSD > > # Use space "data" for data, "meta" for meta-data/cinfo files > pfc.spaces data meta Yes. So... is /xcache-root a real path? Is /xcache-meta a real path? This doesn't help me with what I should point oss.localroot at. Sam > > On 2019-11-14 11:50, Sam Skipsey wrote: > > Hello, > > So, I've tried various interpretations of the documentation, but I'm > > obviously missing something. > > > > I'm trying to configure an Xrootd Proxy Cache, backed by storage > > aggregated into an oss.space. > > > > The cache is configured like: > > > > > > oss.space public /storage0* > > #oss.space public default / > > > > ofs.osslib libXrdPss.so > > pss.cachelib libXrdFileCache.so > > pfc.ram 16g > > all.trace all > > all.log all > > pfs.diskusage 0.90 0.95 > > > > all.export /xroot:/ outplace > > all.export /root:/ outplace > > > > all.export / outplace > > > > pss.origin svr01.beowulf.cluster:1094 > > xrd.allow host *.beowulf.cluster > > > > pfc.blocksize 256k > > > > pfc.spaces public public > > > > > > > > where the /storage0* spaces are all writable and owned by the xrootd > > user (and when we start up the server, it creates "public" directories > > in each of them). > > > > However, whenever I try to access a file through the proxy, it logs > > that it couldn't write the file to the disk (specifying its path as if > > it was going to write it to the storage with no prefix - that is, > > outside of the specified "public" space, in the literal root > > filesystem). > > If this was a non-spaces based system, I'd specify oss.localroot, but > > that doesn't seem to apply to a space? > > I tried adding the outplace directives to the all.exports, but this > > hasn't changed anything. > > > > What am I missing? None of the example configurations I can see with > > oss.spaces seem to need any additional configuration directives... > > > > Sam > > > > ######################################################################## > > Use REPLY-ALL to reply to list > > > > To unsubscribe from the XROOTD-L list, click the following link: > > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 > > > > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-L list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1