Print

---

Hi Patrick,

I don¹t know the answer of the 1st question. For the 2nd question, there is no way in xrootd to do this. So I use a separate help service on the data servers:

So if your copy script do something like "echo filename mtime atime | nc dataserver 5151", then this script will help you set the mtime.

#!/bin/sh

# xinetd service
#service setmtime
#{
#    type        = UNLISTED
#    socket_type = stream
#    protocol    = tcp
#    wait        = no
#    user        = atldq2
#    server      = /u/at/atldq2/bin/setmtime.sh
#    port        = 5151 
#    log_on_failure  = HOST ATTEMPT
#    log_on_success  = 
#    disable     = no 
#}

exec > /dev/null 2>&1
read line
set $line
[ $# -ne 2 -a $# -ne 3 ] && exit
[ ! -f $1 ] && exit
if echo $2 | egrep -q '^[0-9]+$'; then  # set mtime
    touch -m -c -d @$2 $1
fi
[ $# -eq 2 ] && exit
if echo $3 | egrep -q '^[0-9]+$'; then  # set atime
    touch -a -c -d @$3 $1
fi




--
Wei Yang  |  [log in to unmask]  |  650-926-3338(O)







-----Original Message-----
From: <[log in to unmask]> on behalf of Patrick McGuigan <[log in to unmask]>
Date: Friday, March 2, 2018 at 2:00 AM
To: xrootd-l <[log in to unmask]>
Subject: Mass data movement between dataservers

>Hi,
>
>I need to move large numbers of files between dataservers.  In one case this is 
>because I want to retire an existing host and, in another case, I need to free 
>up space on data servers so that overall usage among the dataservers is more 
>balanced.
>
>I can do this with existing scripts that I have used in the past, but there are 
>places I would like to improve things.
>
>The general operation is to:
>1) create a list of files to be moved and a list of dataservers that will accept 
>the copies.
>
>2) Determine the space name associated with each file, so that the destination 
>maintains the space name.
>
>3) In multiple threads copy the file with space information, verify the local 
>and remote checksums, delete the local file.  I normally kick off one process 
>per destination host.
>
>
>I utilize a mapping function that looks at the path of a local file to determine 
>what space name it should belong to.  I am wondering if I can ask the dataserver 
>holding the file, what the space name is.  Is this possible, or if I can make an 
>xrdcp replicate the opaque information.
>
>
>Another place I would like to improve things is to maintain the timestamps 
>associated with the original file, so that the modtime of the copied file 
>matches the modtime of the original file.  Is there there a way of doing the 
>equivalent to:
>
>touch -mt <STAMP> path
>
>on a file stored in xrootd?
>
>
>
>Should I be using FRM to manage all of this?
>
>
>Regards,
>
>Patrick
>
>########################################################################
>Use REPLY-ALL to reply to list
>
>To unsubscribe from the XROOTD-L list, click the following link:
>https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1

########################################################################
Use REPLY-ALL to reply to list

To unsubscribe from the XROOTD-L list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1