Following on chat today in OSG AHM in terms of the XRootd Ceph plugin. Just draft dump of my thoughts and what I have encountered. 

1. We depend fully on OSG XRootD distribution to install, maintain XRootD Servers.

2. OSG does not build the XRootd-Ceph plugin.

3. XRootd-FS would not fit us - as we need to keep and store adler32 checksum. Maybe we could get around this with a separate checksum calculation. Never tried, but I think doable.

4. I cant simply include xrootd-ceph.repo [1] and install xrootd-ceph plugin as I will get many conflicts [2] (I could enforce specific version `yum install xrootd-ceph-4.12.3`, but will be the same issue. 

5. xrootd-ceph plugin from xrootd depends on a specific version of ceph, e.g. 4.12.3 on librados 14.2 (means Nautilus), while cluster I test is Mimic, I also have Nautilus, but the issue remains for me to go away from OSG team build xrootd version to xrootd team build version. If I do so - the only peace of software remaining dependent on the OSG team is condor (So I can change that too and be free from OSG, even this is not what I am tackling and OSG does the hard part to validate all software is ready for HEP).

      a. Ask OSG to change how they name RPMs, so there is no conflict and I can remain using OSG repos? 

      b. Move away from OSG XRootd build to XRootd? For the future, this still needs next point c. 

      c. Any prod ready XRootD release (and a few back releases, 4.11, 4.12, 5.0, and soon 5.1) would be nice if the plugin is compatible with ceph 2 last active releases (Nautilus, Octupus).

Now XrootD-Ceph plugin features needed:

1. Now it defines nbStripes, stripeUnit, objectSize - which can be overwritten during startup. What Ceph allows is to control these parameters per directory. So any time XRootd-Ceph plugin writes data - it should find out directory parameters (nbStripes, stripeUnit, objectSize) and use it. If not defined, use default.

2. That is a futuristic idea: Allow xrootd-ceph to control what EC and what nbStripes, stripeUnit, objectSize to use. In this case, site defines for pool what EC is supported and VO during file put can make decision - use EC 10,1 and stripe unit 16mb and so on. That is just an idea and I have not tested this functionality fully yet - so not sure to what level is possible. 

[1]

[2]