Hi Justas,

Thanks for summarizing the issues. For features in xrootd-ceph, I hope Andy or Alastair know whether there is development resource for that.

From your other points, it seems to me that you want to continue using OSG repo and

I think Brian Lin will have a say on whether OSG can change the naming convention of OSG built Xrootd rpms. I suspect that he does not have such a flexibility.
Can OSG integrate Xrootd-ceph into their xrootd RPM suit? I don’t know whether OSG has the resource to do so.
How much is xrootd-ceph rpms depends on a specific version of CEPH. If repo maintainer has to maintain separate RPMs for each CEPH release, that is a disaster for both repo maintainer and users.

I think you may want use something like yum install –disablerepo=”*” –enablerepo=”osg” … to pick and choose which repo you choose to pick the xrootd release. You may (but not guaranteed) to be able to navigate through those constraints. In some of my cases, I do things in the following order:

Install all other rpms xrootd depended (from EPEL), for example voms
Install xrootd from OSG (or xrootd), specific the version I need.
Continue other rpms installation.

regards,

Wei Yang | [log in to unmask] | 650-926-3338(O)

On 9/4/20, 3:18 PM, "[log in to unmask] on behalf of Justas Balcas" <[log in to unmask] on behalf of [log in to unmask]> wrote:

Hi Folks,

Following on chat today in OSG AHM in terms of the XRootd Ceph plugin. Just draft dump of my thoughts and what I have encountered.

1. We depend fully on OSG XRootD distribution to install, maintain XRootD Servers.

2. OSG does not build the XRootd-Ceph plugin.

3. XRootd-FS would not fit us - as we need to keep and store adler32 checksum. Maybe we could get around this with a separate checksum calculation. Never tried, but I think doable.

4. I cant simply include xrootd-ceph.repo [1] and install xrootd-ceph plugin as I will get many conflicts [2] (I could enforce specific version `yum install xrootd-ceph-4.12.3`, but will be the same issue.

5. xrootd-ceph plugin from xrootd depends on a specific version of ceph, e.g. 4.12.3 on librados 14.2 (means Nautilus), while cluster I test is Mimic, I also have Nautilus, but the issue remains for me to go away from OSG team build xrootd version to xrootd team build version. If I do so - the only peace of software remaining dependent on the OSG team is condor (So I can change that too and be free from OSG, even this is not what I am tackling and OSG does the hard part to validate all software is ready for HEP).

a. Ask OSG to change how they name RPMs, so there is no conflict and I can remain using OSG repos?

b. Move away from OSG XRootd build to XRootd? For the future, this still needs next point c.

c. Any prod ready XRootD release (and a few back releases, 4.11, 4.12, 5.0, and soon 5.1) would be nice if the plugin is compatible with ceph 2 last active releases (Nautilus, Octupus).

Now XrootD-Ceph plugin features needed:

1. Now it defines nbStripes, stripeUnit, objectSize - which can be overwritten during startup. What Ceph allows is to control these parameters per directory. So any time XRootd-Ceph plugin writes data - it should find out directory parameters (nbStripes, stripeUnit, objectSize) and use it. If not defined, use default.

2. That is a futuristic idea: Allow xrootd-ceph to control what EC and what nbStripes, stripeUnit, objectSize to use. In this case, site defines for pool what EC is supported and VO during file put can make decision - use EC 10,1 and stripe unit 16mb and so on. That is just an idea and I have not tested this functionality fully yet - so not sure to what level is possible.

[1]

[root@transfer-5 ~]# cat /etc/yum.repos.d/xrootd.repo

[xrootd-ceph]

name=XRootD Ceph repository

baseurl=http://xrootd.cern.ch/xrootd-ceph-repo/cc-7-x86_64/

gpgcheck=1

enabled=1

protect=0

gpgkey=http://xrootd.cern.ch/sw/releases/RPM-GPG-KEY.txt

[2]

[root@transfer-5 ~]# yum install xrootd-ceph

Loaded plugins: fastestmirror, priorities

Loading mirror speeds from cached hostfile

* base: centos3.zswap.net

* epel: d2lzkl7pfhq30w.cloudfront.net

* extras: mirrors.xtom.com

* osg: mirror.grid.uchicago.edu

* updates: mirrors.xtom.com

56 packages excluded due to repository priority protections

Resolving Dependencies

--> Running transaction check

---> Package xrootd-ceph.x86_64 1:5.0.1-1.el7 will be installed

--> Processing Dependency: xrootd-server-libs(x86-64) = 1:5.0.1-1.el7 for package: 1:xrootd-ceph-5.0.1-1.el7.x86_64

--> Processing Dependency: xrootd-libs(x86-64) = 1:5.0.1-1.el7 for package: 1:xrootd-ceph-5.0.1-1.el7.x86_64

--> Processing Dependency: xrootd-client-libs(x86-64) = 1:5.0.1-1.el7 for package: 1:xrootd-ceph-5.0.1-1.el7.x86_64

--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 1:xrootd-ceph-5.0.1-1.el7.x86_64

--> Processing Dependency: libXrdUtils.so.3()(64bit) for package: 1:xrootd-ceph-5.0.1-1.el7.x86_64

--> Finished Dependency Resolution

Error: Package: 1:xrootd-ceph-5.0.1-1.el7.x86_64 (xrootd-ceph)

Requires: xrootd-libs(x86-64) = 1:5.0.1-1.el7

Installed: 1:xrootd-libs-4.12.3-1.osg35.el7.x86_64 (@osg)

xrootd-libs(x86-64) = 1:4.12.3-1.osg35.el7

Available: 1:xrootd-libs-4.10.0-1.osg35.el7.x86_64 (osg)

xrootd-libs(x86-64) = 1:4.10.0-1.osg35.el7

Available: 1:xrootd-libs-4.10.1-1.osg35.el7.x86_64 (osg)

xrootd-libs(x86-64) = 1:4.10.1-1.osg35.el7

Available: 1:xrootd-libs-4.11.0-1.osg35.el7.x86_64 (osg)

xrootd-libs(x86-64) = 1:4.11.0-1.osg35.el7

Available: 1:xrootd-libs-4.11.1-1.osg35.el7.x86_64 (osg)

xrootd-libs(x86-64) = 1:4.11.1-1.osg35.el7

Available: 1:xrootd-libs-4.11.2-1.osg35.el7.x86_64 (osg)

xrootd-libs(x86-64) = 1:4.11.2-1.osg35.el7

Available: 1:xrootd-libs-4.11.3-1.2.osg35.el7.x86_64 (osg)

xrootd-libs(x86-64) = 1:4.11.3-1.2.osg35.el7

Error: Package: 1:xrootd-ceph-5.0.1-1.el7.x86_64 (xrootd-ceph)

Requires: xrootd-client-libs(x86-64) = 1:5.0.1-1.el7

Installed: 1:xrootd-client-libs-4.12.3-1.osg35.el7.x86_64 (@osg)

xrootd-client-libs(x86-64) = 1:4.12.3-1.osg35.el7

Available: 1:xrootd-client-libs-4.10.0-1.osg35.el7.x86_64 (osg)

xrootd-client-libs(x86-64) = 1:4.10.0-1.osg35.el7

Available: 1:xrootd-client-libs-4.10.1-1.osg35.el7.x86_64 (osg)

xrootd-client-libs(x86-64) = 1:4.10.1-1.osg35.el7

Available: 1:xrootd-client-libs-4.11.0-1.osg35.el7.x86_64 (osg)

xrootd-client-libs(x86-64) = 1:4.11.0-1.osg35.el7

Available: 1:xrootd-client-libs-4.11.1-1.osg35.el7.x86_64 (osg)

xrootd-client-libs(x86-64) = 1:4.11.1-1.osg35.el7

Available: 1:xrootd-client-libs-4.11.2-1.osg35.el7.x86_64 (osg)

xrootd-client-libs(x86-64) = 1:4.11.2-1.osg35.el7

Available: 1:xrootd-client-libs-4.11.3-1.2.osg35.el7.x86_64 (osg)

xrootd-client-libs(x86-64) = 1:4.11.3-1.2.osg35.el7

Error: Package: 1:xrootd-ceph-5.0.1-1.el7.x86_64 (xrootd-ceph)

Requires: libXrdUtils.so.3()(64bit)

Error: Package: 1:xrootd-ceph-5.0.1-1.el7.x86_64 (xrootd-ceph)

Requires: librados.so.2(LIBRADOS_14.2.0)(64bit)

Error: Package: 1:xrootd-ceph-5.0.1-1.el7.x86_64 (xrootd-ceph)

Requires: xrootd-server-libs(x86-64) = 1:5.0.1-1.el7

Installed: 1:xrootd-server-libs-4.12.3-1.osg35.el7.x86_64 (@osg)

xrootd-server-libs(x86-64) = 1:4.12.3-1.osg35.el7

Available: 1:xrootd-server-libs-4.10.0-1.osg35.el7.x86_64 (osg)

xrootd-server-libs(x86-64) = 1:4.10.0-1.osg35.el7

Available: 1:xrootd-server-libs-4.10.1-1.osg35.el7.x86_64 (osg)

xrootd-server-libs(x86-64) = 1:4.10.1-1.osg35.el7

Available: 1:xrootd-server-libs-4.11.0-1.osg35.el7.x86_64 (osg)

xrootd-server-libs(x86-64) = 1:4.11.0-1.osg35.el7

Available: 1:xrootd-server-libs-4.11.1-1.osg35.el7.x86_64 (osg)

xrootd-server-libs(x86-64) = 1:4.11.1-1.osg35.el7

Available: 1:xrootd-server-libs-4.11.2-1.osg35.el7.x86_64 (osg)

xrootd-server-libs(x86-64) = 1:4.11.2-1.osg35.el7

Available: 1:xrootd-server-libs-4.11.3-1.2.osg35.el7.x86_64 (osg)

xrootd-server-libs(x86-64) = 1:4.11.3-1.2.osg35.el7

You could try using --skip-broken to work around the problem

You could try running: rpm -Va --nofiles --nodigest

Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1