 |
 |
Hi Fabrizio, I am trying to understand how far away is XrdHTTP from a S3 object store. I know XrdHttp will work with 'requests', etc. but by far the most popular object store is s3. Rubin is one of the experiments that widely uses s3 (they claim they support webdav as well). They don't use all the features of s3 but a subset of basic functions. Things like ACL and multipart upload are mentioned but not in high priority. I am not sure I fully understand the difference between HTTP(s) and S3. I thought that S3 is built on top of HTTPs. So I am trying boto3 against XrdHTTP. It both surprises and not surprise to me that some of the things are actually working. e.g.download_file(), upload_file() (if I remove the emoji from the XrdHTTP response), get_object(), delete_object(), delete_bucket(). etc. There are things that doesn't work or need twisting. AA is one, which I worked around by using a macaroon token. Listing objects/buckets is another one because boto3 expect some kind of XML (probably specified in S3 protocol). XrdHTTP can identify access via boto3 api by checking the user-agent. The following is what I see in the log file: http_Protocol: rc:91 got hdr line: User-Agent: Boto3/1.12.49 Python/3.6.8 Linux/3.10.0-1160.49.1.el7.x86_64 Botocore/1.15.49 I don't know how far we can and should go along this route. I am assessing what is possible, and what is not. -- Wei ________________________________________ From: Fabrizio Furano <[log in to unmask]> Sent: Monday, April 11, 2022 12:54 AM To: Yang, Wei; xrootd-l Subject: Re: Feature request on XrdHTTP Hi Wei, could you please be more specific about the requests that should get an XML? I would prefer to keep the browsers working against XrdHttp (which I think it's a great feature), so I would not just substitute HTML with XML Can these requests that need XML be recognized through their headers? Also, boto3 uses the AWS protocol, while XrdHttp is ... HTTP. Does not seem the perfect match to me. Maybe we can think at a plugin that provides basic S3 compliance instead than adding tweaks and special cases? Cheers Fabrizio Il 09/04/2022 00:00, Yang, Wei ha scritto: > I am playing with boto3 with XrdHTTP. After a few hack to pass a Macaroon to xrootd I can get boto3 to download a file from xrootd (I guess I can use scitoken too). But list_objects() and list_buckets() do not appreciate the HTML returned by the servers. They want XML. Could this be possible? > > -- > Wei > > ######################################################################## > Use REPLY-ALL to reply to list > > To unsubscribe from the XROOTD-L list, click the following link: > https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1