Yes, as I posted in the original email: "If I switch to 5.5.5 client, there is no issue"
Thanks,
Bockjoo
On 8/29/23 12:11, Brian Lin wrote:
> Hi all,
>
> We're currently seeing similar TLS issues in osg-test with xrdcp. It's
> why we haven't released 5.6.x in the OSG repos yet:
> https://opensciencegrid.atlassian.net/browse/SOFTWARE-5623?focusedCommentId=380033.
>
> I'm also CC'ing Lincoln who is seeing this issue out in the wild with
> a 5.6.1 client against a 5.5.5 server (see attached). For comparison,
> I've also attached a successful copy for a 5.5.5 client vs 5.5.5
> server for the same file.
>
> Bockjoo: would you be able to try downgrading to a 5.5 client and
> seeing if that resolves your issue?
>
> Thanks,
> Brian
>
>
> On 8/28/23 08:42, Bockjoo Kim wrote:
>> Hi Guilherme,
>>
>> The client machine (xrootd 5.6.1) that I had the issue is a fully
>> credible OSG CE.
>>
>> So, there should be no issue with the CA and X509_CERT_DIR.
>>
>> I have used this python script to reproduce the issue on the client
>> machine:
>>
>> #############################################
>>
>> import os
>> import sys
>> import errno
>> import subprocess
>> import zlib
>> import random
>> from XRootD import client
>> from XRootD.client.flags import OpenFlags
>>
>> ENDPOINT='cmsio2.rc.ufl.edu:1094'
>> SAM_TEST_FILE='/store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/A64CCCF2-5C76-E711-B359-0CC47A78A3F8.root'
>>
>>
>> print ("XRootD Client Versin",client.__version__)
>> cmd = [ "xrdfs "+ENDPOINT+" query config version" ]
>> try:
>> result = subprocess.run(cmd, shell=True, capture_output=True,
>> text=True)
>> print("XRootD Server Version", result.stdout)
>> except subprocess.TimeoutExpired:
>> print("connecting to endpoint timed out")
>>
>>
>> os.environ["X509_CERT_DIR"] =
>> "/cvmfs/cms.cern.ch/grid/etc/grid-security/certificates"
>> os.environ["X509_USER_PROXY"] = "/home/bockjoo/.cmsuser.proxy"
>> os.environ["X509_USER_PROXY_NONCMS"] = "/home/bockjoo/.griduser.proxy"
>> os.environ["XRD_NETWORKSTACK"] = "IPv4"
>> with client.File() as f:
>> status, response = f.open("root://" + ENDPOINT + "/" + \
>> SAM_TEST_FILE, flags=OpenFlags.READ, timeout=90)
>> if ( not status.ok ):
>> print (("\nopen(root://%s/%s, flags=OpenFlags.READ,
>> time" + \
>> "out=90)\nXRootDStatus.code=%d \"%s\"\n") % \
>> (ENDPOINT, SAM_TEST_FILE, status.code, \
>> status.message.replace("\n", "")))
>> pass
>> status, data = f.read(offset=0, size=65536, timeout=90)
>> if ( not status.ok ):
>> print(("\n%s\nread(offset=0, size=65536,
>> timeout=90)\n" + \
>> "XRootDStatus.code=%d \"%s\"\n") %
>> (SAM_TEST_FILE, \
>> status.code, status.message.replace("\n",
>> "")))
>> pass
>> print ("Open Status",status.ok)
>>
>> #############################################
>>
>> You can choose the endpoint and the file of your choosing with the
>> 5.5.5 server
>>
>> to test it.
>>
>> Thanks,
>>
>> Bockjoo
>>
>> On 8/28/23 09:33, Guilherme Amadio wrote:
>>> Dear Bockjoo,
>>>
>>> On Sat, Aug 26, 2023 at 04:14:28PM -0400, Bockjoo Kim wrote:
>>>> Hi,
>>>>
>>>> I am seeing a python XRootD file open issue for the 5.6.1 client
>>>> with a
>>>> 5.5.5 server :
>>>>
>>>> =============================================================
>>>>
>>>> XRootD Client Versin 5.6.1
>>>> XRootD Server Version v5.5.5
>>>>
>>>> open(root://cmsio2.rc.ufl.edu:1094//store/mc/SAM/GenericTTbar/AODSIM/CMSSW_9_2_6_91X_mcRun1_realistic_v2-v1/00000/A64CCCF2-5C76-E711-B359-0CC47A78A3F8.root,
>>>>
>>>> flags=OpenFlags.READ, timeout=90)
>>>> XRootDStatus.code=110 "[FATAL] TLS error: resource temporarily
>>>> unavailable: Unable to connect to cmsio2.rc.ufl.edu; error_ssl"
>>>>
>>>> ---------------------------------------------------------------------------
>>>>
>>>> ValueError Traceback (most recent
>>>> call last)
>>>> /tmp/ipykernel_4179061/812350213.py in <module>
>>>> 40 status.message.replace("\n", "")))
>>>> 41 #pass
>>>> ---> 42 status, data = f.read(offset=0, size=65536,
>>>> timeout=90)
>>>> 43 if ( not status.ok ):
>>>> 44 print(("\n%s\nread(offset=0, size=65536,
>>>> timeout=90)\n" + \
>>>>
>>>> /opt/cms/services/anaconda3/lib/python3.9/site-packages/XRootD/client/file.py
>>>>
>>>> in read(self, offset, size, timeout, callback)
>>>> 124 return XRootDStatus(self.__file.read(offset, size,
>>>> timeout, callback))
>>>> 125
>>>> --> 126 status, response = self.__file.read(offset, size, timeout)
>>>> 127 return XRootDStatus(status), response
>>>> 128
>>>>
>>>> ValueError: I/O operation on closed file
>>>>
>>>> ===============================================================
>>>>
>>>> Here, XRootD Server is configured with TLS.
>>>>
>>>> If I remove TLS configuration of the 5.5.5 server, there is no issue.
>>>>
>>>> If I switch to 5.5.5 client, there is no issue.
>>>>
>>>> Is this expected?
>>> It may or may not be. When I wrote the patch, I tested several
>>> scenarios
>>> (see
>>> https://github.com/xrootd/xrootd/pull/2031#issuecomment-1589380486).
>>> The error message that you see is likely caused by a client that cannot
>>> validate the server with TLS (because it does not have the proper CA
>>> certificates installed locally). So I suggest you to try with xrdcp
>>> --notlsok option, or export X509_CERT_DIR=/dev/null to force the client
>>> into not being able to do TLS at all. If the directory
>>> /etc/grid-security
>>> exists on your machine, but the client cannot verify the server, and
>>> TLS
>>> is enforced, then this behavior is expected. Otherwise, please export
>>> XRD_LOGLEVEL=Dump, re-run the command and send us the output so I can
>>> investigate this issue further. You may also want to install the proper
>>> certificates to let the client validate the server to be able to use
>>> TLS.
>>>
>>> Best regards,
>>> -Guilherme
>>
>> ########################################################################
>> Use REPLY-ALL to reply to list
>>
>> To unsubscribe from the XROOTD-L list, click the following link:
>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1
