 |
 |
Andy, What's with the invalid certificate in the client log? [2022-02-15 09:56:19.549139 -0800][Debug ][XRootDTransport ] [transfer-9.ultralight.org:1095.0] Sending out kXR_login request, username: root, cgi: ?xrd.cc=us&xrd.tz=-8&xrd.appname=xrdcp&xrd.info=&xrd.hostname=xrd-cache-3.ultralight.org&xrd.rn=v5.2.0, dual-stack: true, private IPv4: false, private IPv6: false [2022-02-15 09:56:19.549209 -0800][Debug ][AsyncSock ] [transfer-9.ultralight.org:1095.0] TLS hand-shake exchange. ===> HERE: [2022-02-15 09:56:19.551762 -0800][Error ][TlsMsg ] [TLS_Context:] Unable to create TLS context; invalid certificate. [2022-02-15 09:56:19.551903 -0800][Error ][AsyncSock ] [transfer-9.ultralight.org:1095.0] Socket error while handshaking: [FATAL] TLS error [2022-02-15 09:56:19.551920 -0800][Debug ][AsyncSock ] [transfer-9.ultralight.org:1095.0] Closing the socket Can I run in gdb to get more info? What is good place to start poking? I was assuming it's the server cert that client does not like ... but it does look ok to me :) Matevz On 2/15/22 14:04, Andrew Hanushevsky wrote: > Hi Bockjoo, > > Unfortunately, that's not the way it works. While gsi doesn't need to use TLS > ztn does. Since the erver doesn't know which protocol the client will eventually > settle on, the connection has to use TLS right from the start. That means you > cannot use ztn with incapable clients. > > Andy > > > On Tue, 15 Feb 2022, Bockjoo Kim wrote: > >> Hi Andy, >> >> There are two sec.protocols: gsi and ztn. >> >> Doesn't the interaction go through gsi and if it fails, will it go through ztn? >> >> For incapable clients, the gsi can succeed, no? >> >> Thanks, >> >> Bockjoo >> >> On 2/15/22 14:52, Andrew Hanushevsky wrote: >>> Hi Justas, >>> >>> If you look into the log you will notice a warning that tells you that TLS >>> will always be on regardless of the "capable" setting because authentication >>> protocol ztn requires tls. So, this may be the source of the problem, >>> certainly it will be for incapable clients. >>> >>> Andy >>> >>> >>> On Tue, 15 Feb 2022, Justas Balcas wrote: >>> >>>> Hi, >>>> >>>> Server's/Clients are running > 5.3.X release (not 5.4). I took out 1 server >>>> from prod and played with full debug mode, on/off tls. >>>> >>>> Logs from server/client are available here: >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__login-2D1.hep.caltech.edu_-7Ejbalcas_tls_&d=DwIDaQ&c=-35OiAkTchMrZOngvJPOeA&r=f2PhPg2_OoVvPAKGXfp4WG1YcdhQC9qsy2uMHw3Z_6k&m=cxGPTdnshuISdao1V4PsW-krXJuMxpk57T8cXetcP32UCNNa9pgqm--1NTcRuVNW&s=tns3GlZokUnAN9f_4ZHOR1kQXE0DrfqOgDK3bos6dS4&e= >>>> >>>> To turn TLS, I added this config: >>>> xrd.tls /etc/grid-security/xrootd/xrootdcert.pem >>>> /etc/grid-security/xrootd/xrootdkey.pem >>>> xrd.tlsca certdir /etc/grid-security/certificates >>>> xrootd.tls capable all >>>> sec.protocol /usr/lib64 ztn >>>> >>>> And with TLS on - I always get: >>>> TLS hand-shake exchange. >>>> Socket error while handshaking: [FATAL] TLS error >>>> Closing the socket >>>> >>>> >>>> If it helps, here is full config: >>>> >>>> all.export /tmp stage >>>> frm.xfr.copycmd /bin/cp /dev/null $PFN >>>> all.adminpath /var/spool/xrootd >>>> all.pidpath /var/run/xrootd >>>> >>>> # XrootD Security >>>> # --------------------------------------- >>>> xrootd.seclib /usr/lib64/libXrdSec.so >>>> sec.protocol /usr/lib64 gsi -certdir:/etc/grid-security/certificates >>>> -cert:/etc/grid-security/xrootd/xrootdcert.pem >>>> -key:/etc/grid-security/xrootd/xrootdkey.pem -crl:3 >>>> -authzfun:libXrdLcmaps.so -authzto:900 >>>> -authzfunparms:lcmapscfg=/etc/xrootd/lcmaps.cfg -gmapopt:10 -gmapto:0 >>>> acc.authdb /etc/xrootd/auth_file_stageout >>>> ofs.authorize >>>> macaroons.secretkey /etc/xrootd/macaroon-secret >>>> ofs.authlib ++ libXrdMacaroons.so >>>> ofs.authlib ++ libXrdAccSciTokens.so >>>> # -------------------------------------- >>>> # XrootD Monitoring >>>> # -------------------------------------- >>>> # Monitoring for AAA Dashboard : >>>> xrd.report 169.228.130.91:9931 every 30s all sync >>>> xrootd.monitor all auth flush 30s window 5s fstat 60 lfn ops xfr 5 dest >>>> files io info user 169.228.130.91:9930 dest fstat info user >>>> xrd-mon.osgstorage.org:9930 >>>> all.sitename T2_US_Caltech >>>> # ------------------------------------- >>>> # Configure redirector/server >>>> # ------------------------------------- >>>> set xrdr = xrootd-redir-stageout.ultralight.org >>>> xrd.port 1095 >>>> all.manager $(xrdr):1213 >>>> >>>> if $(xrdr) >>>> # It's role is manager >>>> all.role manager >>>> # Redirect all lookup calls to original data servers. Redirector does not >>>> have visibility of FS >>>> cms.dfs lookup distrib mdhold 20m redirect immed >>>> else >>>> # Role is server >>>> all.role server >>>> # The known managers (local redirector) >>>> all.manager meta $(xrdr):1213 >>>> >>>> # Enable xrootd checksum calculation "on-the-fly" using multiuser plugin >>>> # This makes XRootD to write the files with the >>>> # ownership of the user that authenticated to the server and not as the >>>> # 'xrootd' user >>>> ofs.osslib ++ libXrdMultiuser.so >>>> # Enable the checksum wrapper >>>> ofs.ckslib * libXrdMultiuser.so >>>> # Control of checksum >>>> xrootd.chksum max 10 adler32 >>>> multiuser.checksumonwrite on >>>> multiuser.umask 0002 >>>> >>>> fi >>>> # ------------------------------------- >>>> # Allow only specific path, checksum config >>>> # ------------------------------------- >>>> # Allow any path to be exported; this is further refined in the authfile. >>>> all.export / >>>> >>>> # Hosts allowed to use this xrootd cluster >>>> cms.allow host * >>>> >>>> # Enable xrootd debugging >>>> xrootd.trace emsg login stall redirect >>>> cms.trace defer files forward redirect >>>> >>>> # Disable async. Related issue: >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_xrootd_xrootd_issues_1113&d=DwIDaQ&c=-35OiAkTchMrZOngvJPOeA&r=f2PhPg2_OoVvPAKGXfp4WG1YcdhQC9qsy2uMHw3Z_6k&m=cxGPTdnshuISdao1V4PsW-krXJuMxpk57T8cXetcP32UCNNa9pgqm--1NTcRuVNW&s=L2XeYWMRRqstD75CgZb1yCHO9dgWL2K6Uqmqto5rx_Q&e= >>>> xrootd.async off >>>> >>>> # ------------------------------------- >>>> # Integrate with CMS Namespaces >>>> # It will see files under /store/... >>>> # ------------------------------------- >>>> oss.localroot /storage/cms >>>> # ------------------------------------- >>>> # Configure davs/https for TPC >>>> # ------------------------------------- >>>> # Enable https over XrootD >>>> if exec xrootd >>>> xrd.protocol http:1095 /usr/lib64/libXrdHttp.so >>>> http.cadir /etc/grid-security/certificates >>>> http.cert /etc/grid-security/xrootd/xrootdcert.pem >>>> http.key /etc/grid-security/xrootd/xrootdkey.pem >>>> http.secxtractor /usr/lib64/libXrdLcmaps.so >>>> http.secretkey XXXXXXX >>>> # Enable third-party-copy >>>> http.exthandler xrdtpc libXrdHttpTPC.so >>>> # Pass the bearer token to the Xrootd authorization framework. >>>> http.header2cgi Authorization authz >>>> http.listingdeny yes >>>> http.desthttps yes >>>> http.selfhttps2http no >>>> http.staticpreload >>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__static_robots.txt&d=DwIDaQ&c=-35OiAkTchMrZOngvJPOeA&r=f2PhPg2_OoVvPAKGXfp4WG1YcdhQC9qsy2uMHw3Z_6k&m=cxGPTdnshuISdao1V4PsW-krXJuMxpk57T8cXetcP32UCNNa9pgqm--1NTcRuVNW&s=Laz4x7NvD_fTheDJu5m_cEES6_pePEidCLIAkrYNvPs&e= >>>> /etc/xrootd/robots.txt >>>> http.exthandler xrdmacaroons libXrdMacaroons.so >>>> fi >>>> >>>> >>>> xrd.tls /etc/grid-security/xrootd/xrootdcert.pem >>>> /etc/grid-security/xrootd/xrootdkey.pem >>>> xrd.tlsca certdir /etc/grid-security/certificates >>>> xrootd.tls capable all >>>> sec.protocol /usr/lib64 ztn >>>> >>>> >>>> >>>> xrootd.trace all >>>> xrd.trace all >>>> ofs.trace all >>>> pfc.trace all >>>> cms.trace all >>>> # To debug connections to the fedration (5 Dump, 4 Debug, 3 Error, 2 >>>> Warning, 1 Info) >>>> pss.setopt DebugLevel 4 >>>> >>>> ######################################################################## >>>> Use REPLY-ALL to reply to list >>>> >>>> To unsubscribe from the XROOTD-L list, click the following link: >>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIDaQ&c=-35OiAkTchMrZOngvJPOeA&r=f2PhPg2_OoVvPAKGXfp4WG1YcdhQC9qsy2uMHw3Z_6k&m=cxGPTdnshuISdao1V4PsW-krXJuMxpk57T8cXetcP32UCNNa9pgqm--1NTcRuVNW&s=nmhVko_mWVLPvjSwEKtKkm17GDZSKRYlu7FW5xSiWAg&e= >>>> >>> >>> ######################################################################## >>> Use REPLY-ALL to reply to list >>> >>> To unsubscribe from the XROOTD-L list, click the following link: >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIDaQ&c=-35OiAkTchMrZOngvJPOeA&r=f2PhPg2_OoVvPAKGXfp4WG1YcdhQC9qsy2uMHw3Z_6k&m=cxGPTdnshuISdao1V4PsW-krXJuMxpk57T8cXetcP32UCNNa9pgqm--1NTcRuVNW&s=nmhVko_mWVLPvjSwEKtKkm17GDZSKRYlu7FW5xSiWAg&e= >> >> >> ######################################################################## >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-L list, click the following link: >> https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIDaQ&c=-35OiAkTchMrZOngvJPOeA&r=f2PhPg2_OoVvPAKGXfp4WG1YcdhQC9qsy2uMHw3Z_6k&m=cxGPTdnshuISdao1V4PsW-krXJuMxpk57T8cXetcP32UCNNa9pgqm--1NTcRuVNW&s=nmhVko_mWVLPvjSwEKtKkm17GDZSKRYlu7FW5xSiWAg&e= >> > > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-L list, click the following link: > https://urldefense.proofpoint.com/v2/url?u=https-3A__listserv.slac.stanford.edu_cgi-2Dbin_wa-3FSUBED1-3DXROOTD-2DL-26A-3D1&d=DwIDaQ&c=-35OiAkTchMrZOngvJPOeA&r=f2PhPg2_OoVvPAKGXfp4WG1YcdhQC9qsy2uMHw3Z_6k&m=cxGPTdnshuISdao1V4PsW-krXJuMxpk57T8cXetcP32UCNNa9pgqm--1NTcRuVNW&s=nmhVko_mWVLPvjSwEKtKkm17GDZSKRYlu7FW5xSiWAg&e= > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1