 |
 |
I am not sure I understand you. ³sss² is optional on the xrootdfs side. On the xrootd server side, you can have 1) no security 2) only ³sss² security or only ³unix² security 3) either ³sss² or ³unix² security such as this xrootd.seclib /usr/lib64/libXrdSec.so sec.protocol /usr/lib64 sss -s /etc/xrootd/sss.keytab sec.protocol /usr/lib64 unix Will this help? The root cause of why the same procedure (cp via FUSE w/o permission, xrdfs rm, and xrdcp) gave different results to you and to me is still unknown. regards, -- Wei Yang | [log in to unmask] | 650-926-3338(O) -----Original Message----- From: Heiko Schröter <[log in to unmask]> Date: Friday, April 27, 2018 at 12:07 AM To: Wei Yang <[log in to unmask]>, xrootd-l <[log in to unmask]> Cc: "[log in to unmask]" <[log in to unmask]> Subject: Re: xrootd 4.8.3-rc1 FUSE mount "eligible servers shunned" >Sounds like it could be it. >"SSS" keying is not really an option for us because an unaware user >might still mount xrootd FUSE on her own machine without it. >So the root cause would remain to the system. > >Is there anything else i could test because we still have the test setup >in work ? > >Heiko > > > > > >Am 27.04.2018 um 08:53 schrieb Yang, Wei: >> In XrootdFS source code (XrdFfsXrootdfs.cc) I have the following comment which seems to describe >> a scenario that is similar to what you saw. >> >> /* >> Around May 2008, the O_EXCL was added to the _open(). No reason was given. It is removed again >> due to the following reason (the situation that redirector thinks a file exist while it doesn't): >> >> >> 1. FUSE will use _getattr to determine file status. _mknod() will be called only if _getattr() >> determined that the file does not exist. >> 2. In the case that rootd security is enabled, if a user create a file at an unauthorized path >> (and fail), redirector thinks the files exist but it actually does't exist (enabling security >> on redirector doesn't seems to help. An authorized user won't be able to create the same file >> until the redirector forgets about it. >> >> >> res = XrdFfsPosix_open(rootpath, O_CREAT | O_EXCL | O_WRONLY, S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH); >> */ >> >> But the above comment was put there many years ago and thing may have change. And indeed I can not repeat >> the problem now. In my setup which I enabled ?sss? security, is here what I see: >> >> >> >> [scs@griddev01 atlas]$ id >> uid=15001(scs) gid=1051(sf) groups=1051(sf) >> [scs@griddev01 atlas]$ pwd >> /xrootd/atlas >> [scs@griddev01 atlas]$ cp /tmp/README junk >> cp: cannot create regular file `junk': Permission denied >> >> >> >> >> The above show user ?scs? can not create the file due to permission. But below, user ?yangw? rm and create the file. The results are expected. >> >> [yangw@griddev01 atlas]$ xrdfs atlrdr1:11094 rm /xrootd/atlas/junk >> [ERROR] Server responded with an error: [3011] No servers are available to write the file. >> >> [yangw@griddev01 atlas]$ xrdcp /tmp/README root://atlrdr1:11094//xrootd/atlas/junk >> [998B/998B][100%][==================================================][998B/s] >> >> >> >> >> >> >> >> >> -- >> Wei Yang | [log in to unmask] | 650-926-3338(O) >> >> >> >> >> >> >> >> -----Original Message----- >> From: Heiko Schröter <[log in to unmask]> >> Date: Thursday, April 26, 2018 at 11:27 PM >> To: Wei Yang <[log in to unmask]>, xrootd-l <[log in to unmask]> >> Cc: "[log in to unmask]" <[log in to unmask]> >> Subject: Re: xrootd 4.8.3-rc1 FUSE mount "eligible servers shunned" >> >>> No the file does not exist. I checked it on the data servers. >>> >>> The scenario is just as given from step 1 to step 3 in that order. >>> First do a a failed "fuse" copy (permission denied, which is wanted in >>> that dir). >>> Just followed by the other two commands. >>> Nothing in between. >>> >>> Heiko >>> >>> >>> >>> Am 27.04.2018 um 08:23 schrieb Yang, Wei: >>>> Only the first one uses XrootdFS (xrootd fuse mount). for 3) do you know if the file actually exist before xrdcp? If so, you will need -f option to xrdcp. >>>> >>>> -- >>>> Wei Yang | [log in to unmask] | 650-926-3338(O) >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> -----Original Message----- >>>> From: <[log in to unmask]> on behalf of Heiko Schröter <[log in to unmask]> >>>> Date: Thursday, April 26, 2018 at 1:45 AM >>>> To: xrootd-l <[log in to unmask]> >>>> Cc: "[log in to unmask]" <[log in to unmask]> >>>> Subject: xrootd 4.8.3-rc1 FUSE mount "eligible servers shunned" >>>> >>>>> In short: >>>>> 1) cp file via xrootd FUSE mount -> no permission # OK. Wanted behaviour. >>>>> 2) xrdfs rm file -> Unable to write file; eligible servers shunned >>>>> 3) xrdcp file -> Unable to create new file; file already exists. >>>>> >>>>> It does not happen when copying a file with xrdcp into a forbidden >>>>> directory. >>>>> >>>>> Only solution seems to be restarting the redirector. >>>>> This situation may lead into a completely blocked namespace if a user >>>>> starts a mass copy job via FUSE mount (by accident). >>>>> >>>>> Are there any other options ? >>>>> IMHO xrootd should not create any caching issues when a file is >>>>> forbidden to write/create/copy. >>>>> >>>>> >>>>> Heiko >>>>> >>>>> P.S: Also this "stalling client" pops up. >>>>> >>>>> >>>>> >>>>> ++++++++++++++++++++++++++++++++++++++++++++++ >>>>> Resume State after fresh restart of redirector: >>>>> 180426 10:03:53 4079 setStatus REDIRECTOR sent resume event >>>>> 180426 10:03:53 4079 cms_setStatus: Manager REDIRECTOR resumed >>>>> >>>>> >>>>> Copy via FUSE mnt without permissions which is the wanted behaviour: >>>>> schroete@CLIENT:~$ cp MetopA.sh /mnt/xrootd/schroete/willi.dat >>>>> cp: reguläre Datei '/mnt/xrootd/schroete/willi.dat' kann nicht angelegt >>>>> werden: Keine Berechtigung >>>>> i.e. no permissions >>>>> >>>>> ```bash >>>>> 180426 10:15:21 4071 XrdSched: Now have 3 workers >>>>> 180426 10:15:21 4071 XrdSched: running main accept inq=0 >>>>> 180426 10:15:21 4247 XrdXeq: Worker thread started >>>>> 180426 10:15:21 4072 XrdInet: Accepted connection from 7@CLIENT >>>>> 180426 10:15:21 4072 XrdProtocol: matched protocol xrootd >>>>> 180426 10:15:21 4072 ?:7@CLIENT XrdPoll: FD 7 attached to poller 0; num=1 >>>>> 180426 10:15:21 4072 ?:7@CLIENT XrootdProtocol: 0000 req=login dlen=104 >>>>> 180426 10:15:21 4072 root.271390:7@CLIENT XrootdResponse: 0000 sending >>>>> 16 data bytes >>>>> 180426 10:15:21 4072 XrootdXeq: root.271390:7@CLIENT pvt IPv4 login >>>>> 180426 10:15:21 4072 root.271390:7@CLIENT XrootdProtocol: 0100 >>>>> req=locate dlen=8 >>>>> 180426 10:15:21 4072 root.271390:7@CLIENT XrootdProtocol: 0100 locate >>>>> /xrootd/ >>>>> 180426 10:15:21 4072 root.271390:7@CLIENT ofs_fsctl: fn=/xrootd/ >>>>> 180426 10:15:21 4079 Receive glogin1 4 bytes on 3071 >>>>> 180426 10:15:21 4079 Decode glogin1 delays root.271390:7@CLIENT 5 /xrootd/ >>>>> 180426 10:15:21 4072 root.271390:7@CLIENT XrootdProtocol: 0100 rc=5 >>>>> locate /xrootd/ >>>>> 180426 10:15:21 4072 root.271390:7@CLIENT XrootdProtocol: 0100 stalling >>>>> client for 5 sec >>>>> 180426 10:15:21 4072 root.271390:7@CLIENT XrootdResponse: 0100 sending 4 >>>>> data bytes; status=4005 >>>>> 180426 10:15:23 4247 XrdSched: Now have 4 workers >>>>> 180426 10:15:23 4247 XrdSched: running underused thread monitor inq=0 >>>>> 180426 10:15:23 4247 XrdSched: 4 threads; 0 idle >>>>> 180426 10:15:23 4247 XrdSched: scheduling underused thread monitor in >>>>> 780 seconds >>>>> 180426 10:15:23 4249 XrdXeq: Worker thread started >>>>> 180426 10:15:24 4072 root.271390:7@CLIENT XrootdProtocol: 0100 request >>>>> timeout; read 0 of 24 bytes >>>>> 180426 10:15:24 4072 XrdPoll: Poller 0 enabled root.271390:7@CLIENT >>>>> 180426 10:15:26 4247 XrdSched: running root.271390:7@CLIENT inq=0 >>>>> 180426 10:15:26 4247 root.271390:7@CLIENT XrootdProtocol: 0100 >>>>> req=locate dlen=8 >>>>> 180426 10:15:26 4247 root.271390:7@CLIENT XrootdProtocol: 0100 locate >>>>> /xrootd/ >>>>> 180426 10:15:26 4247 root.271390:7@CLIENT ofs_fsctl: fn=/xrootd/ >>>>> 180426 10:15:26 4079 Receive glogin1 315 bytes on 4095 >>>>> 180426 10:15:26 4079 Decode glogin1 sent root.271390:7@CLIENT >>>>> 'Sw[::192.168.16.146]:1094 Sw[::192.168.16.144]:1094 >>>>> Sw[::192.168.16.127]:1094 Sw[::192.168.16.120]:1094 >>>>> Sw[::192.168.16.147]:1094 Sw[::192.168.16.139]:1094 >>>>> Sw[::192.168.16.97]:1094 Sw[::192.168.16.195]:1094 >>>>> Sw[::192.168.16.196]:1094 Sw[::192.168.16.134]:1094 >>>>> Sw[::192.168.16.121]:1094 Sw[::192.168.16.217]:1094' /xrootd/ >>>>> 180426 10:15:26 4247 root.271390:7@CLIENT XrootdProtocol: 0100 rc=-1024 >>>>> locate /xrootd/ >>>>> 180426 10:15:26 4247 root.271390:7@CLIENT XrootdResponse: 0100 sending >>>>> 311 data bytes >>>>> 180426 10:15:26 4249 XrdSched: running main accept inq=0 >>>>> 180426 10:15:26 4071 XrdInet: Accepted connection from 19@CLIENT >>>>> 180426 10:15:26 4071 XrdProtocol: matched protocol xrootd >>>>> 180426 10:15:26 4071 ?:19@CLIENT XrdPoll: FD 19 attached to poller 1; num=1 >>>>> 180426 10:15:26 4071 ?:19@CLIENT XrootdProtocol: 0000 req=login dlen=104 >>>>> 180426 10:15:26 4071 h21.271390:19@CLIENT XrootdResponse: 0000 sending >>>>> 16 data bytes >>>>> 180426 10:15:26 4071 XrootdXeq: h21.271390:19@CLIENT pvt IPv4 login >>>>> 180426 10:15:26 4071 h21.271390:19@CLIENT XrootdProtocol: 0100 req=open >>>>> dlen=27 >>>>> 180426 10:15:26 4071 h21.271390:19@CLIENT XrootdProtocol: 0100 open >>>>> udmat /xrootd//schroete/willi.dat >>>>> 180426 10:15:26 4071 h21.271390:19@CLIENT ofs_open: 200-40644 >>>>> fn=/xrootd/schroete/willi.dat >>>>> 180426 10:15:26 4079 Receive glogin1 4 bytes on 5119 >>>>> 180426 10:15:26 4079 Decode glogin1 delays h21.271390:19@CLIENT 5 >>>>> /xrootd/schroete/willi.dat >>>>> 180426 10:15:26 4071 h21.271390:19@CLIENT XrootdProtocol: 0100 stalling >>>>> client for 5 sec >>>>> 180426 10:15:26 4071 h21.271390:19@CLIENT XrootdResponse: 0100 sending 4 >>>>> data bytes; status=4005 >>>>> 180426 10:15:26 4071 h21.271390:19@CLIENT ofs_close: use=0 fn=dummy >>>>> 180426 10:15:29 4247 root.271390:7@CLIENT XrootdProtocol: 0100 request >>>>> timeout; read 0 of 24 bytes >>>>> 180426 10:15:29 4247 XrdPoll: Poller 0 enabled root.271390:7@CLIENT >>>>> 180426 10:15:29 4071 h21.271390:19@CLIENT XrootdProtocol: 0100 request >>>>> timeout; read 0 of 24 bytes >>>>> 180426 10:15:29 4071 XrdPoll: Poller 1 enabled h21.271390:19@CLIENT >>>>> 180426 10:15:31 4072 XrdSched: running h21.271390:19@CLIENT inq=0 >>>>> 180426 10:15:31 4072 h21.271390:19@CLIENT XrootdProtocol: 0100 req=open >>>>> dlen=27 >>>>> 180426 10:15:31 4072 h21.271390:19@CLIENT XrootdProtocol: 0100 open >>>>> udmat /xrootd//schroete/willi.dat >>>>> 180426 10:15:31 4072 h21.271390:19@CLIENT ofs_open: 200-40644 >>>>> fn=/xrootd/schroete/willi.dat >>>>> 180426 10:15:31 4079 Receive glogin1 19 bytes on 6143 >>>>> 180426 10:15:31 4079 Decode glogin1 redirects h21.271390:19@CLIENT to >>>>> 192.168.16.217:1094 /xrootd/schroete/willi.dat >>>>> 180426 10:15:31 4072 h21.271390:19@CLIENT XrootdProtocol: 0100 >>>>> redirecting to 192.168.16.217:1094 >>>>> 180426 10:15:31 4072 h21.271390:19@CLIENT XrootdResponse: 0100 sending >>>>> 18 data bytes; status=4004 >>>>> 180426 10:15:31 4072 h21.271390:19@CLIENT ofs_close: use=0 fn=dummy >>>>> 180426 10:15:34 4072 h21.271390:19@CLIENT XrootdProtocol: 0100 request >>>>> timeout; read 0 of 24 bytes >>>>> 180426 10:15:34 4072 XrdPoll: Poller 1 enabled h21.271390:19@CLIENT >>>>> ``` >>>>> >>>>> Trying to remove the file: >>>>> schroete@CLIENT:~$ xrdfs glogin1 rm /xrootd/schroete/willi.dat >>>>> [ERROR] Server responded with an error: [3011] Unable to write file; >>>>> eligible servers shunned. >>>>> >>>>> ```bash >>>>> 180426 10:20:15 4247 XrdSched: running main accept inq=0 >>>>> 180426 10:20:15 4249 XrdInet: Accepted connection from 18@CLIENT >>>>> 180426 10:20:15 4249 XrdProtocol: matched protocol xrootd >>>>> 180426 10:20:15 4249 ?:18@CLIENT XrdPoll: FD 18 attached to poller 2; num=1 >>>>> 180426 10:20:15 4249 ?:18@CLIENT XrootdProtocol: 0000 req=login dlen=101 >>>>> 180426 10:20:15 4249 schroete.271507:18@CLIENT XrootdResponse: 0000 >>>>> sending 16 data bytes >>>>> 180426 10:20:15 4249 XrootdXeq: schroete.271507:18@CLIENT pvt IPv4 login >>>>> 180426 10:20:15 4249 schroete.271507:18@CLIENT XrootdProtocol: 0100 >>>>> req=rm dlen=26 >>>>> 180426 10:20:15 4249 schroete.271507:18@CLIENT ofs_remove: f >>>>> fn=/xrootd/schroete/willi.dat >>>>> 180426 10:20:15 4079 Receive glogin1 19 bytes on 7167 >>>>> 180426 10:20:15 4079 Decode glogin1 redirects schroete.271507:18@CLIENT >>>>> to 192.168.16.217:1094 /xrootd/schroete/willi.dat >>>>> 180426 10:20:15 4249 schroete.271507:18@CLIENT XrootdProtocol: 0100 >>>>> rc=-256 rm /xrootd/schroete/willi.dat >>>>> 180426 10:20:15 4249 schroete.271507:18@CLIENT XrootdProtocol: 0100 >>>>> redirecting to 192.168.16.217:1094 >>>>> 180426 10:20:15 4249 schroete.271507:18@CLIENT XrootdResponse: 0100 >>>>> sending 18 data bytes; status=4004 >>>>> 180426 10:20:15 4249 schroete.271507:18@CLIENT XrootdProtocol: 0100 >>>>> req=rm dlen=62 >>>>> 180426 10:20:15 4249 schroete.271507:18@CLIENT ofs_remove: f >>>>> fn=/xrootd/schroete/willi.dat >>>>> 180426 10:20:15 4079 Receive glogin1 52 bytes on 8191 >>>>> 180426 10:20:15 4079 Decode glogin1 gave schroete.271507:18@CLIENT err >>>>> -2 'Unable to write file; eligible servers shunned.' >>>>> /xrootd/schroete/willi.dat >>>>> 180426 10:20:15 4249 schroete.271507:18@CLIENT XrootdProtocol: 0100 >>>>> rc=-1 rm /xrootd/schroete/willi.dat >>>>> 180426 10:20:15 4249 schroete.271507:18@CLIENT XrootdResponse: 0100 >>>>> sending err 3011: Unable to write file; eligible servers shunned. >>>>> 180426 10:20:15 4249 XrootdXeq: schroete.271507:18@CLIENT disc 0:00:00 >>>>> 180426 10:20:15 4249 schroete.271507:18@CLIENT XrdPoll: FD 18 detached >>>> >from poller 2; num=0 >>>>> ``` >>>>> >>>>> >>>>> Tryint to copy with xrdcp tool: >>>>> schroete@CLIENT:~$ xrdcp MetopA.sh >>>>> xroot://glogin1//xrootd/schroete/willi.dat >>>>> [0B/0B][100%][==================================================][0B/s] >>>>> Run: [ERROR] Server responded with an error: [3011] Unable to create new >>>>> file; file already exists.180426 >>>>> >>>>> ```bash >>>>> 180426 10:21:03 4071 XrdSched: running h21.271390:19@CLIENT inq=0 >>>>> 180426 10:21:03 4071 XrootdXeq: h21.271390:19@CLIENT disc 0:05:37 >>>>> 180426 10:21:03 4071 h21.271390:19@CLIENT XrdPoll: FD 19 detached from >>>>> poller 1; num=0 >>>>> 180426 10:21:03 4072 XrdSched: running root.271390:7@CLIENT inq=0 >>>>> 180426 10:21:03 4072 XrootdXeq: root.271390:7@CLIENT disc 0:05:42 >>>>> 180426 10:21:03 4072 root.271390:7@CLIENT XrdPoll: FD 7 detached from >>>>> poller 0; num=0 >>>>> 180426 10:21:05 4247 XrdInet: Accepted connection from 21@CLIENT >>>>> 180426 10:21:05 4247 XrdProtocol: matched protocol xrootd >>>>> 180426 10:21:05 4247 ?:21@CLIENT XrdPoll: FD 21 attached to poller 0; num=1 >>>>> 180426 10:21:05 4247 ?:21@CLIENT XrootdProtocol: 0000 req=login dlen=101 >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdResponse: 0000 >>>>> sending 16 data bytes >>>>> 180426 10:21:05 4247 XrootdXeq: schroete.271528:21@CLIENT pvt IPv4 login >>>>> 180426 10:21:05 4249 XrdSched: running main accept inq=0 >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdProtocol: 0100 >>>>> req=stat dlen=26 >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT ofs_stat: >>>>> fn=/xrootd/schroete/willi.dat >>>>> 180426 10:21:05 4079 Receive glogin1 19 bytes on 9215 >>>>> 180426 10:21:05 4079 Decode glogin1 redirects schroete.271528:21@CLIENT >>>>> to 192.168.16.217:1094 /xrootd/schroete/willi.dat >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdProtocol: 0100 >>>>> rc=-256 stat /xrootd/schroete/willi.dat >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdProtocol: 0100 >>>>> redirecting to 192.168.16.217:1094 >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdResponse: 0100 >>>>> sending 18 data bytes; status=4004 >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdProtocol: 0100 >>>>> req=stat dlen=62 >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT ofs_stat: >>>>> fn=/xrootd/schroete/willi.dat >>>>> 180426 10:21:05 4079 Receive glogin1 47 bytes on 10239 >>>>> 180426 10:21:05 4079 Decode glogin1 gave schroete.271528:21@CLIENT err >>>>> -2 'No servers are available to read the file.' /xrootd/schroete/willi.dat >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdProtocol: 0100 >>>>> rc=-1 stat /xrootd/schroete/willi.dat >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdResponse: 0100 >>>>> sending err 3011: No servers are available to read the file. >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdProtocol: 0100 >>>>> req=open dlen=40 >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdProtocol: 0100 open >>>>> unmat /xrootd/schroete/willi.dat?oss.asize=226 >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT ofs_open: 102-40644 >>>>> fn=/xrootd/schroete/willi.dat >>>>> 180426 10:21:05 4079 Receive glogin1 4 bytes on 11263 >>>>> 180426 10:21:05 4079 Decode glogin1 delays schroete.271528:21@CLIENT 5 >>>>> /xrootd/schroete/willi.dat >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdProtocol: 0100 >>>>> stalling client for 5 sec >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT XrootdResponse: 0100 >>>>> sending 4 data bytes; status=4005 >>>>> 180426 10:21:05 4247 schroete.271528:21@CLIENT ofs_close: use=0 fn=dummy >>>>> 180426 10:21:08 4247 schroete.271528:21@CLIENT XrootdProtocol: 0100 >>>>> request timeout; read 0 of 24 bytes >>>>> 180426 10:21:08 4247 XrdPoll: Poller 0 enabled schroete.271528:21@CLIENT >>>>> 180426 10:21:10 4071 XrdSched: running schroete.271528:21@CLIENT inq=0 >>>>> 180426 10:21:10 4071 schroete.271528:21@CLIENT XrootdProtocol: 0100 >>>>> req=open dlen=40 >>>>> 180426 10:21:10 4071 schroete.271528:21@CLIENT XrootdProtocol: 0100 open >>>>> unmat /xrootd/schroete/willi.dat?oss.asize=226 >>>>> 180426 10:21:10 4071 schroete.271528:21@CLIENT ofs_open: 102-40644 >>>>> fn=/xrootd/schroete/willi.dat >>>>> 180426 10:21:10 4079 Receive glogin1 52 bytes on 12287 >>>>> 180426 10:21:10 4079 Decode glogin1 gave schroete.271528:21@CLIENT err >>>>> -2 'Unable to create new file; file already exists.' >>>>> /xrootd/schroete/willi.dat >>>>> 180426 10:21:10 4071 schroete.271528:21@CLIENT XrootdResponse: 0100 >>>>> sending err 3011: Unable to create new file; file already exists. >>>>> 180426 10:21:10 4071 schroete.271528:21@CLIENT ofs_close: use=0 fn=dummy >>>>> 180426 10:21:10 4071 XrootdXeq: schroete.271528:21@CLIENT disc 0:00:05 >>>>> 180426 10:21:10 4071 schroete.271528:21@CLIENT XrdPoll: FD 21 detached >>>> >from poller 0; num=0 >>>>> 180426 10:22:23 4069 XrdBuffManager: Reshaper has 3K; target 403877K >>>>> ``` >>>>> >>>>> ######################################################################## >>>>> Use REPLY-ALL to reply to list >>>>> >>>>> To unsubscribe from the XROOTD-L list, click the following link: >>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >>>> ######################################################################## >>>> Use REPLY-ALL to reply to list >>>> >>>> To unsubscribe from the XROOTD-L list, click the following link: >>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1 >>>> >>> -- >>> ----------------------------------------------------------------------- >>> Dipl.-Ing. Heiko Schröter >>> Institute of Environmental Physics (IUP) phone: ++49-(0)421-218-62092 >>> Institute of Remote Sensing (IFE) fax: ++49-(0)421-218-62070 >>> University of Bremen (FB1) >>> P.O. Box 330440 email: [log in to unmask] >>> Otto-Hahn-Allee 1 >>> 28359 Bremen >>> Germany >>> ----------------------------------------------------------------------- >>> > >-- >----------------------------------------------------------------------- >Dipl.-Ing. Heiko Schröter >Institute of Environmental Physics (IUP) phone: ++49-(0)421-218-62092 >Institute of Remote Sensing (IFE) fax: ++49-(0)421-218-62070 >University of Bremen (FB1) >P.O. Box 330440 email: [log in to unmask] >Otto-Hahn-Allee 1 >28359 Bremen >Germany >----------------------------------------------------------------------- > ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-L list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-L&A=1