Hi Justas,

If you look into the log you will notice a warning that tells you that TLS
will always be on regardless of the "capable" setting because
authentication protocol ztn requires tls. So, this may be the source of
the problem, certainly it will be for incapable clients.

Andy


On Tue, 15 Feb 2022, Justas Balcas wrote:

> Hi,
>
> Server's/Clients are running > 5.3.X release (not 5.4). I took out 1 server
> from prod and played with full debug mode, on/off tls.
>
> Logs from server/client are available here:
> https://login-1.hep.caltech.edu/~jbalcas/tls/
>
> To turn TLS, I added this config:
> xrd.tls /etc/grid-security/xrootd/xrootdcert.pem
> /etc/grid-security/xrootd/xrootdkey.pem
> xrd.tlsca certdir /etc/grid-security/certificates
> xrootd.tls capable all
> sec.protocol /usr/lib64 ztn
>
> And with TLS on - I always get:
> TLS hand-shake exchange.
> Socket error while handshaking: [FATAL] TLS error
> Closing the socket
>
>
> If it helps, here is full config:
>
> all.export /tmp stage
> frm.xfr.copycmd /bin/cp /dev/null $PFN
> all.adminpath /var/spool/xrootd
> all.pidpath /var/run/xrootd
>
> # XrootD Security
> # ---------------------------------------
> xrootd.seclib /usr/lib64/libXrdSec.so
> sec.protocol /usr/lib64 gsi -certdir:/etc/grid-security/certificates
> -cert:/etc/grid-security/xrootd/xrootdcert.pem
> -key:/etc/grid-security/xrootd/xrootdkey.pem -crl:3
> -authzfun:libXrdLcmaps.so -authzto:900
> -authzfunparms:lcmapscfg=/etc/xrootd/lcmaps.cfg -gmapopt:10 -gmapto:0
> acc.authdb /etc/xrootd/auth_file_stageout
> ofs.authorize
> macaroons.secretkey /etc/xrootd/macaroon-secret
> ofs.authlib ++ libXrdMacaroons.so
> ofs.authlib ++ libXrdAccSciTokens.so
> # --------------------------------------
> # XrootD Monitoring
> # --------------------------------------
> # Monitoring for AAA Dashboard :
> xrd.report 169.228.130.91:9931 every 30s all sync
> xrootd.monitor all auth flush 30s window 5s fstat 60 lfn ops xfr 5 dest
> files io info user 169.228.130.91:9930 dest fstat info user
> xrd-mon.osgstorage.org:9930
> all.sitename T2_US_Caltech
> # -------------------------------------
> # Configure redirector/server
> # -------------------------------------
> set xrdr = xrootd-redir-stageout.ultralight.org
> xrd.port 1095
> all.manager $(xrdr):1213
>
> if $(xrdr)
>  # It's role is manager
>  all.role manager
>  # Redirect all lookup calls to original data servers. Redirector does not
> have visibility of FS
>  cms.dfs lookup distrib mdhold 20m redirect immed
> else
>  # Role is server
>  all.role server
>  # The known managers (local redirector)
>  all.manager meta $(xrdr):1213
>
>  # Enable xrootd checksum calculation "on-the-fly" using multiuser plugin
>  # This makes XRootD to write the files with the
>  # ownership of the user that authenticated to the server and not as the
>  # 'xrootd' user
>  ofs.osslib ++ libXrdMultiuser.so
>  # Enable the checksum wrapper
>  ofs.ckslib * libXrdMultiuser.so
>  # Control of checksum
>  xrootd.chksum max 10 adler32
>  multiuser.checksumonwrite on
>  multiuser.umask 0002
>
> fi
> # -------------------------------------
> # Allow only specific path, checksum config
> # -------------------------------------
> # Allow any path to be exported; this is further refined in the authfile.
> all.export /
>
> # Hosts allowed to use this xrootd cluster
> cms.allow host *
>
> # Enable xrootd debugging
> xrootd.trace emsg login stall redirect
> cms.trace defer files forward redirect
>
> # Disable async. Related issue: https://github.com/xrootd/xrootd/issues/1113
> xrootd.async off
>
> # -------------------------------------
> # Integrate with CMS Namespaces
> # It will see files under /store/...
> # -------------------------------------
> oss.localroot /storage/cms
> # -------------------------------------
> # Configure davs/https for TPC
> # -------------------------------------
> # Enable https over XrootD
> if exec xrootd
>  xrd.protocol http:1095 /usr/lib64/libXrdHttp.so
>  http.cadir /etc/grid-security/certificates
>  http.cert /etc/grid-security/xrootd/xrootdcert.pem
>  http.key /etc/grid-security/xrootd/xrootdkey.pem
>  http.secxtractor /usr/lib64/libXrdLcmaps.so
>  http.secretkey XXXXXXX
>  # Enable third-party-copy
>  http.exthandler xrdtpc libXrdHttpTPC.so
>  # Pass the bearer token to the Xrootd authorization framework.
>  http.header2cgi Authorization authz
>  http.listingdeny yes
>  http.desthttps yes
>  http.selfhttps2http no
>  http.staticpreload http://static/robots.txt /etc/xrootd/robots.txt
>  http.exthandler xrdmacaroons libXrdMacaroons.so
> fi
>
>
> xrd.tls /etc/grid-security/xrootd/xrootdcert.pem
> /etc/grid-security/xrootd/xrootdkey.pem
> xrd.tlsca certdir /etc/grid-security/certificates
> xrootd.tls capable all
> sec.protocol /usr/lib64 ztn
>
>
>
> xrootd.trace all
> xrd.trace    all
> ofs.trace    all
> pfc.trace    all
> cms.trace    all
> # To debug connections to the fedration (5 Dump, 4 Debug, 3 Error, 2
> Warning, 1 Info)
> pss.setopt   DebugLevel 4
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
>