I can't explain exactly what is going on other than defining all three
envars leads to a conflict. Other peope have seen this. I can't say it's
new because we didn't have TLS until recently so it may have been the case
all along and now with TLS you see the conflict. In general CERT/KEY
shouldn't be defined when you are using a proxy. I would say it's an
OpenSSL issue where is mismatches the cert with the proxy key or the key
with the proxy cert (don't know which way). Of course, generating a proxy
certificate using any standard tool writes it to /tmp with a well known
name and does not set PROXY and that works just fine as well.
On Wed, 15 Feb 2023, Albert Rossi wrote:
> Hi,
>
> I have noted this behavior which I do not really grasp.
>
> [arossi@fndcatemp1 ~]$ /opt/xrootd/v5.x/bin/xrdcp -version
> v5.5.1
>
> dCache endpoint fndcadoor01.fnal.gov:1094 enforces TLS
>
> using GSI to this endpoint, it would seem that defining CERT, KEY and PROXY leads to the client generating the following error:
>
> ```
> / Load the private key
> //
> if (SSL_CTX_use_PrivateKey_file(pImpl->ctx, key, SSL_FILETYPE_PEM) != 1 )
> FATAL_SSL("Unable to create TLS context; invalid private key.");
>
> ```
>
> Here is how.
>
> 1. Using generated proxy and the KEY/CERT env vars:
>
> [arossi@fndcatemp1 ~]$ voms-proxy-destroy
> [arossi@fndcatemp1 ~]$ export X509_USER_KEY=/home/arossi/.globus/tmp/u8773.key.pem
> [arossi@fndcatemp1 ~]$ export X509_USER_CERT=/home/arossi/.globus/tmp/u8773.crt.pem
> [arossi@fndcatemp1 ~]$ xrdcp5x data/data_1b xroot://fndcadoor01.fnal.gov:1094//pnfs/fnal.gov/usr/eagle/dcache-tests/scratch/als-data-`suffix`
>
> 230215 17:15:18 24961 cryptossl_X509CreateProxy: Your identity: /DC=org/DC=cilogon/C=US/O=Fermi National Accelerator Laboratory/OU=People/CN=Al Rossi/CN=UID:arossi
> [1B/1B][100%][==================================================][1B/s]
>
> Success
>
> 2. Using out-of-band proxy from voms-proxy-init and no env vars:
>
> [arossi@fndcatemp1 ~]$ voms-proxy-destroy
> [arossi@fndcatemp1 ~]$ voms-proxy-init
> Your identity: /DC=org/DC=cilogon/C=US/O=Fermi National Accelerator Laboratory/OU=People/CN=Al Rossi/CN=UID:arossi
> Creating proxy ................................................................................................................... Done
>
> Your proxy is valid until Thu Feb 16 05:16:11 2023
> [arossi@fndcatemp1 ~]$ unset X509_USER_KEY
> [arossi@fndcatemp1 ~]$ unset X509_USER_CERT
> [arossi@fndcatemp1 ~]$ xrdcp5x data/data_1b xroot://fndcadoor01.fnal.gov:1094//pnfs/fnal.gov/usr/eagle/dcache-tests/scratch/als-data-`suffix`
> [1B/1B][100%][==================================================][1B/s]
>
> Success
>
> 3. Defining X509_USER_PROXY with the proxy in that location, but no KEY or CERT:
>
> [arossi@fndcatemp1 ~]$ export X509_USER_PROXY=/tmp/x509up_u8773
> [arossi@fndcatemp1 ~]$ xrdcp5x data/data_1b xroot://fndcadoor01.fnal.gov:1094//pnfs/fnal.gov/usr/eagle/dcache-tests/scratch/als-data-`suffix`
> [1B/1B][100%][==================================================][1B/s]
>
> Success
>
> 4. Now, adding the CERT and KEY env vars
>
> [arossi@fndcatemp1 ~]$ export X509_USER_KEY=/home/arossi/.globus/tmp/u8773.key.pem
> [arossi@fndcatemp1 ~]$ export X509_USER_CERT=/home/arossi/.globus/tmp/u8773.crt.pem
> [arossi@fndcatemp1 ~]$ xrdcp5x data/data_1b xroot://fndcadoor01.fnal.gov:1094//pnfs/fnal.gov/usr/eagle/dcache-tests/scratch/als-data-`suffix`
> [0B/0B][100%][==================================================][0B/s]
> Run: [FATAL] TLS error: Unable to create TLS context; invalid private key. (destination)
>
> 5. Removing the voms proxy, but leaving all variables defined.
>
> [arossi@fndcatemp1 ~]$ xrdcp5x data/data_1b xroot://fndcadoor01.fnal.gov:1094//pnfs/fnal.gov/usr/eagle/dcache-tests/scratch/als-data-`suffix`
> [0B/0B][100%][==================================================][0B/s]
> Run: [FATAL] TLS error: Unable to use cert file /tmp/x509up_u8773; does not exist. (destination)
>
> Failure expected ... no proxy in PROXY loc. BUT:
>
> 6. Regenerating proxy from KEY and CERT and first unsetting the PROXY var:
>
> [arossi@fndcatemp1 ~]$ unset X509_USER_PROXY
> [arossi@fndcatemp1 ~]$ xrdcp5x data/data_1b xroot://fndcadoor01.fnal.gov:1094//pnfs/fnal.gov/usr/eagle/dcache-tests/scratch/als-data-`suffix`
> 230215 17:23:53 32354 cryptossl_X509CreateProxy: Your identity: /DC=org/DC=cilogon/C=US/O=Fermi National Accelerator Laboratory/OU=People/CN=Al Rossi/CN=UID:arossi
> [1B/1B][100%][==================================================][1B/s]
>
> Success. But now, pointing the PROXY var at it (and leaving KEY and CERT defined):
>
> [arossi@fndcatemp1 ~]$ ls -l /tmp/x509up_u8773
> -rw------- 1 arossi ods 3564 Feb 15 17:23 /tmp/x509up_u8773
>
> [arossi@fndcatemp1 ~]$ export X509_USER_PROXY=/tmp/x509up_u8773
> [arossi@fndcatemp1 ~]$ xrdcp5x data/data_1b xroot://fndcadoor01.fnal.gov:1094//pnfs/fnal.gov/usr/eagle/dcache-tests/scratch/als-data-`suffix`
> [0B/0B][100%][==================================================][0B/s]
> Run: [FATAL] TLS error: Unable to create TLS context; invalid private key. (destination)
>
> So, to summarize, whether the actual proxy is generated from the KEY/CERT or generated by voms-proxy-init, the client produces the TLS error if all three env vars are defined. This didn't seem to be the case before.
>
> Could someone kindly explain why this happens?
>
> Thank you,
>
> Al
> ________________________________________________
> Albert L. Rossi
> Senior Software Developer
> Scientific Computing Division, Scientific Data Services, Distributed Data Development
> WH 566
> Fermi National Accelerator Laboratory
> Batavia, IL 60510
> (630) 840-3023
>
>
> ########################################################################
> Use REPLY-ALL to reply to list
>
> To unsubscribe from the XROOTD-L list, click the following link:
> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
>
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
