Hi Justas,
Thanks for summarizing the issues. For features in xrootd-ceph, I hope Andy or Alastair know whether there is development resource for that.
From your other points, it seems to me that you want to continue using OSG repo and
I think you may want use something like yum install –disablerepo=”*” –enablerepo=”osg” … to pick and choose which repo you choose to pick the xrootd release. You may (but not guaranteed) to be able to navigate through those constraints.
In some of my cases, I do things in the following order:
regards,
--
Wei Yang |
[log in to unmask] | 650-926-3338(O)
On 9/4/20, 3:18 PM, "[log in to unmask] on behalf of Justas Balcas" <[log in to unmask] on behalf of
[log in to unmask]> wrote:
Hi Folks,
Following on chat today in OSG AHM in terms of the XRootd Ceph plugin. Just draft dump of my thoughts and what I have encountered.
1. We depend fully on OSG XRootD distribution to install, maintain XRootD Servers.
2. OSG does not build the XRootd-Ceph plugin.
3. XRootd-FS would not fit us - as we need to keep and store adler32 checksum. Maybe we could get around this with a separate checksum calculation. Never tried, but I think doable.
4. I cant simply include xrootd-ceph.repo [1] and install xrootd-ceph plugin as I will get many conflicts [2] (I could enforce specific version `yum install xrootd-ceph-4.12.3`, but will be the same issue.
5. xrootd-ceph plugin from xrootd depends on a specific version of ceph, e.g. 4.12.3 on librados 14.2 (means Nautilus), while cluster I test is Mimic, I also have Nautilus, but the issue remains for me to go away from OSG team build xrootd
version to xrootd team build version. If I do so - the only peace of software remaining dependent on the OSG team is condor (So I can change that too and be free from OSG, even this is not what I am tackling and OSG does the hard part to validate all software
is ready for HEP).
a. Ask OSG to change how they name RPMs, so there is no conflict and I can remain using OSG repos?
b. Move away from OSG XRootd build to XRootd? For the future, this still needs next point c.
c. Any prod ready XRootD release (and a few back releases, 4.11, 4.12, 5.0, and soon 5.1) would be nice if the plugin is compatible with ceph 2 last active releases (Nautilus, Octupus).
Now XrootD-Ceph plugin features needed:
1. Now it defines nbStripes, stripeUnit, objectSize - which can be overwritten during startup. What Ceph allows is to control these parameters per directory. So any time XRootd-Ceph plugin writes data - it should find out directory parameters
(nbStripes, stripeUnit, objectSize) and use it. If not defined, use default.
2. That is a futuristic idea: Allow xrootd-ceph to control what EC and what nbStripes, stripeUnit, objectSize to use. In this case, site defines for
pool what EC is supported and VO during file put can make decision - use EC 10,1 and stripe unit 16mb and so on. That is just an idea and I have not tested this functionality fully yet - so not sure to what level is possible.
[1]
[root@transfer-5 ~]# cat /etc/yum.repos.d/xrootd.repo
[xrootd-ceph]
name=XRootD Ceph repository
gpgcheck=1
enabled=1
protect=0
[2]
[root@transfer-5 ~]# yum install xrootd-ceph
Loaded plugins: fastestmirror, priorities
Loading mirror speeds from cached hostfile
* base: centos3.zswap.net
* epel: d2lzkl7pfhq30w.cloudfront.net
* extras: mirrors.xtom.com
* osg: mirror.grid.uchicago.edu
* updates: mirrors.xtom.com
56 packages excluded due to repository priority protections
Resolving Dependencies
--> Running transaction check
---> Package xrootd-ceph.x86_64 1:5.0.1-1.el7 will be installed
--> Processing Dependency: xrootd-server-libs(x86-64) = 1:5.0.1-1.el7 for package: 1:xrootd-ceph-5.0.1-1.el7.x86_64
--> Processing Dependency: xrootd-libs(x86-64) = 1:5.0.1-1.el7 for package: 1:xrootd-ceph-5.0.1-1.el7.x86_64
--> Processing Dependency: xrootd-client-libs(x86-64) = 1:5.0.1-1.el7 for package: 1:xrootd-ceph-5.0.1-1.el7.x86_64
--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 1:xrootd-ceph-5.0.1-1.el7.x86_64
--> Processing Dependency: libXrdUtils.so.3()(64bit) for package: 1:xrootd-ceph-5.0.1-1.el7.x86_64
--> Finished Dependency Resolution
Error: Package: 1:xrootd-ceph-5.0.1-1.el7.x86_64 (xrootd-ceph)
Requires: xrootd-libs(x86-64) = 1:5.0.1-1.el7
Installed: 1:xrootd-libs-4.12.3-1.osg35.el7.x86_64 (@osg)
xrootd-libs(x86-64) = 1:4.12.3-1.osg35.el7
Available: 1:xrootd-libs-4.10.0-1.osg35.el7.x86_64 (osg)
xrootd-libs(x86-64) = 1:4.10.0-1.osg35.el7
Available: 1:xrootd-libs-4.10.1-1.osg35.el7.x86_64 (osg)
xrootd-libs(x86-64) = 1:4.10.1-1.osg35.el7
Available: 1:xrootd-libs-4.11.0-1.osg35.el7.x86_64 (osg)
xrootd-libs(x86-64) = 1:4.11.0-1.osg35.el7
Available: 1:xrootd-libs-4.11.1-1.osg35.el7.x86_64 (osg)
xrootd-libs(x86-64) = 1:4.11.1-1.osg35.el7
Available: 1:xrootd-libs-4.11.2-1.osg35.el7.x86_64 (osg)
xrootd-libs(x86-64) = 1:4.11.2-1.osg35.el7
Available: 1:xrootd-libs-4.11.3-1.2.osg35.el7.x86_64 (osg)
xrootd-libs(x86-64) = 1:4.11.3-1.2.osg35.el7
Error: Package: 1:xrootd-ceph-5.0.1-1.el7.x86_64 (xrootd-ceph)
Requires: xrootd-client-libs(x86-64) = 1:5.0.1-1.el7
Installed: 1:xrootd-client-libs-4.12.3-1.osg35.el7.x86_64 (@osg)
xrootd-client-libs(x86-64) = 1:4.12.3-1.osg35.el7
Available: 1:xrootd-client-libs-4.10.0-1.osg35.el7.x86_64 (osg)
xrootd-client-libs(x86-64) = 1:4.10.0-1.osg35.el7
Available: 1:xrootd-client-libs-4.10.1-1.osg35.el7.x86_64 (osg)
xrootd-client-libs(x86-64) = 1:4.10.1-1.osg35.el7
Available: 1:xrootd-client-libs-4.11.0-1.osg35.el7.x86_64 (osg)
xrootd-client-libs(x86-64) = 1:4.11.0-1.osg35.el7
Available: 1:xrootd-client-libs-4.11.1-1.osg35.el7.x86_64 (osg)
xrootd-client-libs(x86-64) = 1:4.11.1-1.osg35.el7
Available: 1:xrootd-client-libs-4.11.2-1.osg35.el7.x86_64 (osg)
xrootd-client-libs(x86-64) = 1:4.11.2-1.osg35.el7
Available: 1:xrootd-client-libs-4.11.3-1.2.osg35.el7.x86_64 (osg)
xrootd-client-libs(x86-64) = 1:4.11.3-1.2.osg35.el7
Error: Package: 1:xrootd-ceph-5.0.1-1.el7.x86_64 (xrootd-ceph)
Requires: libXrdUtils.so.3()(64bit)
Error: Package: 1:xrootd-ceph-5.0.1-1.el7.x86_64 (xrootd-ceph)
Requires: librados.so.2(LIBRADOS_14.2.0)(64bit)
Error: Package: 1:xrootd-ceph-5.0.1-1.el7.x86_64 (xrootd-ceph)
Requires: xrootd-server-libs(x86-64) = 1:5.0.1-1.el7
Installed: 1:xrootd-server-libs-4.12.3-1.osg35.el7.x86_64 (@osg)
xrootd-server-libs(x86-64) = 1:4.12.3-1.osg35.el7
Available: 1:xrootd-server-libs-4.10.0-1.osg35.el7.x86_64 (osg)
xrootd-server-libs(x86-64) = 1:4.10.0-1.osg35.el7
Available: 1:xrootd-server-libs-4.10.1-1.osg35.el7.x86_64 (osg)
xrootd-server-libs(x86-64) = 1:4.10.1-1.osg35.el7
Available: 1:xrootd-server-libs-4.11.0-1.osg35.el7.x86_64 (osg)
xrootd-server-libs(x86-64) = 1:4.11.0-1.osg35.el7
Available: 1:xrootd-server-libs-4.11.1-1.osg35.el7.x86_64 (osg)
xrootd-server-libs(x86-64) = 1:4.11.1-1.osg35.el7
Available: 1:xrootd-server-libs-4.11.2-1.osg35.el7.x86_64 (osg)
xrootd-server-libs(x86-64) = 1:4.11.2-1.osg35.el7
Available: 1:xrootd-server-libs-4.11.3-1.2.osg35.el7.x86_64 (osg)
xrootd-server-libs(x86-64) = 1:4.11.3-1.2.osg35.el7
You could try using --skip-broken to work around the problem
You could try running: rpm -Va --nofiles --nodigest
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1