I'm currently seeing a hard to understand error with an internal Xrootd proxy cache.

Normally, the cache has functioned as expected - purging when it gets above highWatermark [and returning XrdFileCache_File: error File::Open() Create failed , err_code=28, err_str=No space left on device (and forwarding requests) while it does so, until it frees up below lowWatermark again.

The cache uses xrootd spaces to manage its backend disk storage, which is distributed across 6 volumes, all of which are SSDs.

However, we currently see the same behaviour on the cache today, with all 6 filesystems in the cache only ~70% full [our high and low watermarks are 95% and 90%]; and this has been ongoing for much longer than the refresh time for the cache's metrics.

I can write to the filesystems involved, so it isn't a problem with the underlying storage.

We are on 5.0.3 . 

Does anyone have any suggestions? The purpose of this cache is primarily to smooth and consolidate load, so it forwarding most of its requests is, whilst bearable in the short term, not really good for our performance or reliability.

Yours,

Sam