|
FYI. The next generation of the TFile plug-in made it to ROOT, both 5.34
and 6 branches and will be enabled by default.
This also registers the 'xroot://' protocol to point to this plug-in
instead of the default TNetFile (root protocol). So, we no longer have
the situation where people using 'xroot://' protocol prefix in ROOT use
root instead of xroot protocol to access their files.
Cheers,
Lukasz
-------- Original Message --------
Subject: Re: [root.git 1/2] next generation xrootd client (libNetxNG).
Requires xrootd >= 3.3.5.
Date: Mon, 9 Dec 2013 21:41:47 +0100
From: Fons Rademakers <[log in to unmask]>
Organization: CERN
To: <[log in to unmask]>
Hi Lukasz,
ok, the new client is now part of ROOT (v5-34-00-patches and master
(aka ROOT 6)). Please from now on patches against this code. I've not
tested it yet. The module is turned on by default in the rootrc file in
case xrootd 3.3.5 or better is found at configure time.
Could you test it hard and let me know how it works.
Cheers, Fons.
On 09/12/2013 21:30, Fons Rademakers wrote:
> Commit
> fc05b7c99b98d917157ca777884ee97c0237a70a
> <http://root.cern.ch/gitweb?p=root.git;a=commitdiff;h=fc05b7c99b98d917157ca777884ee97c0237a70a>
> Branch
> master
> Author
> Fons Rademakers <[log in to unmask]>
> Date
> Mon, 9 Dec 2013 20:55:40 +0100
> Message
> next generation xrootd client (libNetxNG). Requires xrootd >= 3.3.5.
>
> Provided by Justin Salmon, Lukasz Janyst (CERN/IT).
>
> * Changed file Makefile <#Makefile>
> * Changed file config/Makefile.depend <#config/Makefile.depend>
> * Changed file config/Makefile.in <#config/Makefile.in>
> * Changed file config/rootrc.in <#config/rootrc.in>
> * Changed file configure <#configure>
> * Changed file etc/plugins/TFile/P100_TXNetFile.C
> <#etc/plugins/TFile/P100_TXNetFile.C>
> * Changed file etc/plugins/TFileStager/P010_TXNetFileStager.C
> <#etc/plugins/TFileStager/P010_TXNetFileStager.C>
> * Changed file etc/plugins/TSystem/P040_TXNetSystem.C
> <#etc/plugins/TSystem/P040_TXNetSystem.C>
> * Changed file io/io/src/TFile.cxx <#io/io/src/TFile.cxx>
> * Changed file net/netx/Module.mk <#net/netx/Module.mk>
> * Added file net/netxng/CMakeLists.txt <#net/netxng/CMakeLists.txt>
> * Added file net/netxng/Module.mk <#net/netxng/Module.mk>
> * Added file net/netxng/inc/LinkDef.h <#net/netxng/inc/LinkDef.h>
> * Added file net/netxng/inc/TNetXNGFile.h <#net/netxng/inc/TNetXNGFile.h>
> * Added file net/netxng/inc/TNetXNGFileStager.h
> <#net/netxng/inc/TNetXNGFileStager.h>
> * Added file net/netxng/inc/TNetXNGSystem.h <#net/netxng/inc/TNetXNGSystem.h>
> * Added file net/netxng/src/TNetXNGFile.cxx <#net/netxng/src/TNetXNGFile.cxx>
> * Added file net/netxng/src/TNetXNGFileStager.cxx
> <#net/netxng/src/TNetXNGFileStager.cxx>
> * Added file net/netxng/src/TNetXNGSystem.cxx
> <#net/netxng/src/TNetXNGSystem.cxx>
>
>
> Changed file Makefile
> <http://root.cern.ch/gitweb?p=root.git;f=Makefile;h=7ec0442;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 275 275 endif
> 276 276 ifeq ($(HASXRD),yes)
> 277 277 MODULES += net/netx
> 278 ifeq ($(BUILDNETXNG),yes)
> 279 MODULES += net/netxng
> 280 endif
> 278 281 ifeq ($(BUILDALIEN),yes)
> 279 282 MODULES += net/alien
> 280 283 endif
> …
> 305 308
> rootx net/rootd io/dcache io/chirp hist/hbook graf2d/asimage \
> 306 309 net/ldap net/krb5auth net/rpdutils net/globusauth \
> 307 310
> bindings/pyroot bindings/ruby io/gfal misc/minicern \
> 308 graf2d/qt gui/qtroot gui/qtgsi net/netx net/alien \
> 311
> graf2d/qt gui/qtroot gui/qtgsi net/netx net/netxng net/alien \
> 309 312
> proof/proofd proof/proofx proof/pq2 graf3d/x3d net/davix \
> 310 313 sql/oracle io/xmlparser math/mathmore \
> 311 314
> tmva math/genetic io/hdfs graf2d/fitsio roofit/roofitcore \
>
>
> Changed file config/Makefile.depend
> <http://root.cern.ch/gitweb?p=root.git;f=config/Makefile.depend;h=0009b8a;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 117 117 SQLLIBDEPM = $(NETLIB) $(IOLIB)
> 118 118 ODBCLIBDEPM = $(NETLIB) $(IOLIB)
> 119 119 NETXLIBDEPM = $(NETLIB) $(IOLIB) $(THREADLIB)
> 120 NETXNGLIBDEPM = $(NETLIB) $(IOLIB) $(THREADLIB)
> 120 121 ORACLELIBDEPM = $(NETLIB) $(IOLIB)
> 121 122 CASTORLIBDEPM = $(NETLIB) $(IOLIB)
> 122 123 PGSQLLIBDEPM = $(NETLIB) $(IOLIB)
> …
> 220 221 SQLLIBDEP = $(SQLLIBDEPM)
> 221 222 ODBCLIBDEP = $(ODBCLIBDEPM)
> 222 223 NETXLIBDEP = $(NETXLIBDEPM)
> 224 NETXNGLIBDEP = $(NETXNGLIBDEPM)
> 223 225 ORACLELIBDEP = $(ORACLELIBDEPM)
> 224 226 CASTORLIBDEP = $(CASTORLIBDEPM)
> 225 227 PGSQLLIBDEP = $(PGSQLLIBDEPM)
> …
> 386 388 SQLLIBEXTRA = lib/libNet.lib lib/libRIO.lib
> 387 389 ODBCLIBEXTRA = lib/libNet.lib lib/libRIO.lib
> 388 390
> NETXLIBEXTRA = lib/libNet.lib lib/libRIO.lib lib/libThread.lib
> 391
> NETXNGLIBEXTRA = lib/libNet.lib lib/libRIO.lib lib/libThread.lib
> 389 392 ORACLELIBEXTRA = lib/libNet.lib lib/libRIO.lib
> 390 393 CASTORLIBEXTRA = lib/libNet.lib lib/libRIO.lib
> 391 394 PGSQLLIBEXTRA = lib/libNet.lib lib/libRIO.lib
> …
> 519 522 SQLLIBEXTRA = -Llib -lNet -lRIO
> 520 523 ODBCLIBEXTRA = -Llib -lNet -lRIO
> 521 524 NETXLIBEXTRA = -Llib -lNet -lRIO -lThread
> 525 NETXNGLIBEXTRA = -Llib -lNet -lRIO -lThread
> 522 526 ORACLELIBEXTRA = -Llib -lNet -lRIO
> 523 527 CASTORLIBEXTRA = -Llib -lNet -lRIO
> 524 528 PGSQLLIBEXTRA = -Llib -lNet -lRIO
>
>
> Changed file config/Makefile.in
> <http://root.cern.ch/gitweb?p=root.git;f=config/Makefile.in;h=7e0c9dd;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 316 316 XRDVERSION := @xrdversion@
> 317 317 HASXRDUTILS := @hasxrdutils@
> 318 318 XRDINCPRIVATE := @xrdincprivate@
> 319 BUILDNETXNG := @buildnetxng@
> 319 320
> 320 321 SRPLIBDIR := @srplibdir@
> 321 322 SRPLIB := @srplib@
>
>
> Changed file config/rootrc.in
> <http://root.cern.ch/gitweb?p=root.git;f=config/rootrc.in;h=1533383;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 54 54 Root.TMemStat.buffersize: 100000
> 55 55 Root.TMemStat.maxcalls: 5000000
> 56 56 #Root.TMemStat.system: gnubuiltin
> 57 Root.TMemStat.system:
> 57 Root.TMemStat.system:
> 58 58
> 59 59
> # Activate memory statistics (size and cnt is used to trap allocation of
> 60 60 # blocks of a certain size after cnt times).
> …
> 505 505 # supported by the underlying TFile implementation. Default is yes.
> 506 506 #TFile.AsyncReading: no
> 507 507
> 508
> # Control the usage of asynchronous prefetching capabilities irrespective
> 508
> # Control the usage of asynchronous prefetching capabilities irrespective
> 509 509 # of the TFile implementation. By default it is disabled.
> 510 510 #TFile.AsyncPrefetching: no
> 511 511
> …
> 632 632
> # wait request can be before the client declares an error.
> 633 633 # XNet.TransactionTimeout: 28800
> 634 634
> 635
> # The following env vars are handled by TNetXNGFile (new xrootd client interface)
> 636 # and related classes (module netxng, libNetxNG.so).
> 637
> 638 # Use the old xrootd client (libNetx) instead of the new (libNetxNG)
> 639
> 640 XNet.UseOldClient: @useoldnetx@
> 641
> 642
> # NetXNG.ConnectionWindow - A time window for the connection establishment. A
> 643
> # connection failure is declared if the connection
> 644
> # is not established within the time window. If a
> 645
> # connection failure happens earlier then another
> 646
> # connection attempt will only be made at the
> 647 # beginning of the next window.
> 648
> # NetXNG.ConnectionRetry - Number of connection attempts that should be
> 649
> # made (number of available connection windows)
> 650 # before declaring a permanent failure.
> 651
> # NetXNG.RequestTimeout - Default value for the time after which an error
> 652
> # is declared if it was impossible to get a
> 653 # response to a request.
> 654 # NetXNG.SubStreamsPerChannel - Number of streams per session.
> 655
> # NetXNG.TimeoutResolution - Resolution for the timeout events. Ie. timeout
> 656 # events will be processed only every
> 657 # XRD_TIMEOUTRESOLUTION seconds.
> 658
> # NetXNG.StreamErrorWindow - Time after which the permanent failure flags are
> 659 # cleared out and a new connection may be
> 660 # attempted if needed.
> 661
> # NetXNG.RunForkHandler - Determines whether the fork handlers should be
> 662 # enabled, making the API fork safe.
> 663 # NetXNG.RedirectLimit - Maximum number of allowed redirections.
> 664
> # NetXNG.WorkerThreads - Number of threads processing user callbacks.
> 665
> # NetXNG.CPChunkSize - Size of a single data chunk handled by xrdcopy.
> 666
> # NetXNG.CPParallelChunks - Maximum number of asynchronous requests being
> 667
> # processed by the xrdcopy command at any given
> 668 # time.
> 669
> # NetXNG.PollerPreference - A comma separated list of poller implementations
> 670 # in order of preference. The default is:
> 671 # libevent,built-in.
> 672 # NetXNG.ClientMonitor - Path to the client monitor library.
> 673
> # NetXNG.ClientMonitorParam - Additional optional parameters that will be
> 674
> # passed to the monitoring object on initialization.
> 675
> 635 676 # Example of custom setting for the Rint application (root.exe).
> 636 677
> # This overrides the default specified above for a generic application.
> 637 678 # Color 5 is yellow.
>
>
> Changed file configure
> <http://root.cern.ch/gitweb?p=root.git;f=configure;h=6705b27;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 5090 5090 fi
> 5091 5091 fi
> 5092 5092 fi
> 5093
> 5094 #
> 5095
> # Build the new client interface (libNetxNG) only for xrootd >= 3.3.5
> 5096 decrefng=300030005
> 5097 buildnetxng="no"
> 5098 useoldnetx="yes"
> 5099
> message "Checking whether new xrootd client (libNetxNG) should be build"
> 5100 if [ $decver -ge $decrefng ] ; then
> 5101 buildnetxng="yes"
> 5102 useoldnetx="no"
> 5103 fi
> 5104 result $buildnetxng
> 5093 5105 fi
> 5094 5106
> 5095 5107 check_explicit "$enable_xrootd" "$enable_xrootd_explicit" \
> …
> 7343 7355 -e "s|@afdsmgrdapmon@|$afdsmgrdapmon|" \
> 7344 7356 -e "s|@buildglite@|$enable_glite|" \
> 7345 7357 -e "s|@buildcocoa@|$enable_cocoa|" \
> 7358 -e "s|@buildnetxng@|$buildnetxng|" \
> 7346 7359 -e "s|@c++11@|$enable_cxx11|" \
> 7347 7360 -e "s|@libc++@|$enable_libcxx|" \
> 7348 7361 < Makefile.tmp > $MAKEOUT
> …
> 7541 7554 -e "s|@iconpath@|$iconpath|" \
> 7542 7555 -e "s|@hasxrd@|$hasxrd|" \
> 7543 7556 -e "s|@hasnotxrd@|$hasnotxrd|" \
> 7557 -e "s|@useoldnetx@|$useoldnetx|" \
> 7544 7558 < rootrc.tmp > $ROOTRCOUT
> 7545 7559 rm -f rootrc.tmp
> 7546 7560 result "done"
>
>
> Changed file etc/plugins/TFile/P100_TXNetFile.C
> <http://root.cern.ch/gitweb?p=root.git;f=etc/plugins/TFile/P100_TXNetFile.C;h=59d52df;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1 1 void P100_TXNetFile()
> 2 2 {
> 3 gPluginMgr->AddHandler("TFile", "^root:", "TXNetFile",
> 4
> "Netx", "TXNetFile(const char*,Option_t*,const char*,Int_t,Int_t,Bool_t)");
> 5 gPluginMgr->AddHandler("TFile", "^xroot:", "TXNetFile",
> 6
> "Netx", "TXNetFile(const char*,Option_t*,const char*,Int_t,Int_t,Bool_t)");
> 3 if (!gEnv->GetValue( "XNet.UseOldClient", 0)) {
> 4 gPluginMgr->AddHandler("TFile", "^[x]?root:", "TNetXNGFile",
> 5
> "NetXNG", "TNetXNGFile(const char*,Option_t*,const char*,Int_t,Int_t,Bool_t)");
> 6 } else {
> 7 gPluginMgr->AddHandler("TFile", "^[x]?root:", "TXNetFile",
> 8
> "Netx", "TXNetFile(const char*,Option_t*,const char*,Int_t,Int_t,Bool_t)");
> 9 }
> 7 10 }
>
>
> Changed file etc/plugins/TFileStager/P010_TXNetFileStager.C
> <http://root.cern.ch/gitweb?p=root.git;f=etc/plugins/TFileStager/P010_TXNetFileStager.C;h=884ce7e;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1 1 void P010_TXNetFileStager()
> 2 2 {
> 3 gPluginMgr->AddHandler("TFileStager", "^root:", "TXNetFileStager",
> 4 "Netx", "TXNetFileStager(const char *)");
> 3 if (!gEnv->GetValue("XNet.UseOldClient", 0)) {
> 4
> gPluginMgr->AddHandler("TFileStager", "^[x]?root:", "TNetXNGFileStager",
> 5 "NetXNG", "TNetXNGFileStager(const char *)");
> 6 } else {
> 7 gPluginMgr->AddHandler("TFileStager", "^[x]?root:", "TXNetFileStager",
> 8 "Netx", "TXNetFileStager(const char *)");
> 9 }
> 5 10 }
>
>
> Changed file etc/plugins/TSystem/P040_TXNetSystem.C
> <http://root.cern.ch/gitweb?p=root.git;f=etc/plugins/TSystem/P040_TXNetSystem.C;h=5a1ccd0;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1 1 void P040_TXNetSystem()
> 2 2 {
> 3 gPluginMgr->AddHandler("TSystem", "^root:", "TXNetSystem",
> 4 "Netx", "TXNetSystem(const char *,Bool_t)");
> 3 if (!gEnv->GetValue("XNet.UseOldClient", 0)) {
> 4 gPluginMgr->AddHandler("TSystem", "^[x]?root:", "TNetXNGSystem",
> 5 "NetXNG", "TNetXNGSystem(const char *,Bool_t)");
> 6 } else {
> 7 gPluginMgr->AddHandler("TSystem", "^[x]?root:", "TXNetSystem",
> 8 "Netx", "TXNetSystem(const char *,Bool_t)");
> 9 }
> 5 10 }
>
>
> Changed file io/io/src/TFile.cxx
> <http://root.cern.ch/gitweb?p=root.git;f=io/io/src/TFile.cxx;h=f071126;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 4020 4020 if (type == kNet) {
> 4021 4021 // Network files
> 4022 4022
> if ((h = gROOT->GetPluginManager()->FindHandler("TFile", name)) &&
> 4023
> !strcmp(h->GetClass(),"TXNetFile") && h->LoadPlugin() == 0) {
> 4023 (!strcmp(h->GetClass(),"TXNetFile")
> || !strcmp(h->GetClass(),"TNetXNGFile"))
> 4024 && h->LoadPlugin() == 0) {
> 4024 4025
> f = (TFile*) h->ExecPlugin(6, name.Data(), option, ftitle, compress, netopt, kTRUE);
> 4025 4026 notfound = kFALSE;
> 4026 4027 }
>
>
> Changed file net/netx/Module.mk
> <http://root.cern.ch/gitweb?p=root.git;f=net/netx/Module.mk;h=eb59983;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 37 37 INCLUDEFILES += $(NETXDEP)
> 38 38 endif
> 39 39
> 40 # When using an external XROOTD distribution XROOTDDIRI and XROOTDDIRL
> 41 # are undefined and have to point to the specified inc and lib dirs.
> 42 ifneq ($(XRDINCDIR),)
> 43 ifeq ($(XROOTDDIRI),)
> 44 XROOTDDIRI := $(XRDINCDIR)
> 45 endif
> 46 endif
> 47 ifneq ($(XRDLIBDIR),)
> 48 ifeq ($(XROOTDDIRL),)
> 49 XROOTDDIRL := $(XRDLIBDIR)
> 50 endif
> 51 endif
> 52
> 53 40 # Xrootd includes
> 54 NETXINCEXTRA := $(XROOTDDIRI:%=-I%)
> 41 NETXINCEXTRA := $(XRDINCDIR:%=-I%)
> 55 42 ifneq ($(EXTRA_XRDFLAGS),)
> 56 43 NETXINCEXTRA += -I$(ROOT_SRCDIR)/proof/proofd/inc
> 57 44 endif
> …
> 61 48
> 62 49 # Xrootd client libs
> 63 50 ifeq ($(PLATFORM),win32)
> 64 NETXLIBEXTRA += $(XROOTDDIRL)/libXrdClient.lib
> 51 NETXLIBEXTRA += $(XRDLIBDIR)/libXrdClient.lib
> 65 52 else
> 66 53 ifeq ($(HASXRDUTILS),no)
> 67 NETXLIBEXTRA += $(XROOTDDIRL) -lXrdOuc -lXrdSys -lXrdClient -lpthread
> 54 NETXLIBEXTRA += $(XRDLIBDIR) -lXrdOuc -lXrdSys -lXrdClient -lpthread
> 68 55 else
> 69 NETXLIBEXTRA += $(XROOTDDIRL) -lXrdUtils -lXrdClient
> 56 NETXLIBEXTRA += $(XRDLIBDIR) -lXrdUtils -lXrdClient
> 70 57 endif
> 71 58 endif
> 72 59
> …
> 76 63 include/%.h: $(NETXDIRI)/%.h $(XROOTDMAKE)
> 77 64 cp $< $@
> 78 65
> 79 $(NETXLIB): $(NETXO) $(NETXDO) $(ORDER_) $(MAINLIBS) $(NETXLIBDEP) \
> 80 $(XRDNETXD)
> 66 $(NETXLIB): $(NETXO) $(NETXDO) $(ORDER_) $(MAINLIBS) $(NETXLIBDEP)
> 81 67 @$(MAKELIB) $(PLATFORM) $(LD) "$(LDFLAGS)" \
> 82 68 "$(SOFLAGS)" libNetx.$(SOEXT) $@ "$(NETXO) $(NETXDO)" \
> 83 69 "$(NETXLIBEXTRA)"
> …
> 85 71 $(call pcmrule,NETX)
> 86 72 $(noop)
> 87 73
> 88 $(NETXDS): $(NETXH) $(NETXL) $(XROOTDMAKE) $(ROOTCINTTMPDEP)
> $(XRDPLUGINS) $(call pcmdep,NETX)
> 74 $(NETXDS): $(NETXH) $(NETXL) $(XROOTDMAKE) $(ROOTCINTTMPDEP)
> 89 75 $(MAKEDIR)
> 90 76 @echo "Generating dictionary $@..."
> 91 77
> $(ROOTCINTTMP) -f $@ $(call dictModule,NETX) -c $(NETXINCEXTRA) $(NETXH) $(NETXL)
>
>
> Added file net/netxng/CMakeLists.txt
> <http://root.cern.ch/gitweb?p=root.git;f=net/netxng/CMakeLists.txt;h=2720c1e;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1
> ############################################################################
> 2 # CMakeLists.txt file for building ROOT net/netxng package
> 3 # @author Lukasz Janyst <[log in to unmask]>
> 4
> ############################################################################
> 5
> 6 include_directories(${XROOTD_INCLUDE_DIR})
> 7 add_definitions(${XROOTD_CFLAGS})
> 8
> 9 ROOT_GENERATE_DICTIONARY(G__NetxNG *.h LINKDEF LinkDef.h)
> 10 ROOT_GENERATE_ROOTMAP(NetxNG LINKDEF LinkDef.h DEPENDENCIES RIO Thread)
> 11
> ROOT_LINKER_LIBRARY(NetxNG *.cxx G__NetxNG.cxx LIBRARIES ${XROOTD_LIBRARIES} DEPENDENCIES RIO Thread)
> 12
> 13 ROOT_INSTALL_HEADERS()
>
>
> Added file net/netxng/Module.mk
> <http://root.cern.ch/gitweb?p=root.git;f=net/netxng/Module.mk;h=f0247fa;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1 # Module.mk for xrd module
> 2 # Copyright (c) 2000 Rene Brun and Fons Rademakers
> 3 #
> 4 # Author: Lukasz Janyst 11/01/2013
> 5
> 6 MODNAME := netxng
> 7 MODDIR := $(ROOT_SRCDIR)/net/$(MODNAME)
> 8 MODDIRS := $(MODDIR)/src
> 9 MODDIRI := $(MODDIR)/inc
> 10
> 11 NETXNGDIR := $(MODDIR)
> 12 NETXNGDIRS := $(MODDIR)/src
> 13 NETXNGDIRI := $(MODDIR)/inc
> 14
> 15 ##### libNetxNG #####
> 16 NETXNGL := $(MODDIRI)/LinkDef.h
> 17 NETXNGDS := $(call stripsrc,$(MODDIRS)/G__NetxNG.cxx)
> 18 NETXNGDO := $(NETXNGDS:.cxx=.o)
> 19 NETXNGDH := $(NETXNGDS:.cxx=.h)
> 20
> 21
> NETXNGH := $(filter-out $(MODDIRI)/LinkDef%,$(wildcard $(MODDIRI)/*.h))
> 22
> NETXNGS := $(filter-out $(MODDIRS)/G__%,$(wildcard $(MODDIRS)/*.cxx))
> 23 NETXNGO := $(call stripsrc,$(NETXNGS:.cxx=.o))
> 24
> 25 NETXNGDEP := $(NETXNGO:.o=.d) $(NETXNGDO:.o=.d)
> 26
> 27 NETXNGLIB := $(LPATH)/libNetxNG.$(SOEXT)
> 28 NETXNGMAP := $(NETXNGLIB:.$(SOEXT)=.rootmap)
> 29
> 30 # used in the main Makefile
> 31 ALLHDRS += $(patsubst $(MODDIRI)/%.h,include/%.h,$(NETXNGH))
> 32 ALLLIBS += $(NETXNGLIB)
> 33 ALLMAPS += $(NETXNGMAP)
> 34
> 35 # include all dependency files
> 36 INCLUDEFILES += $(NETXNGDEP)
> 37
> 38 # Xrootd includes
> 39 NETXNGINCEXTRA := $(XRDINCDIR:%=-I%)
> 40
> 41 # Xrootd client libs
> 42 ifeq ($(PLATFORM),win32)
> 43 NETXNGLIBEXTRA += $(XRDLIBDIR)/libXrdCl.lib
> 44 else
> 45 NETXNGLIBEXTRA += $(XRDLIBDIR) -lXrdUtils -lXrdCl
> 46 endif
> 47
> 48 ##### local rules #####
> 49 .PHONY: all-$(MODNAME) clean-$(MODNAME) distclean-$(MODNAME)
> 50
> 51 include/%.h: $(NETXNGDIRI)/%.h
> 52 cp $< $@
> 53
> 54
> $(NETXNGLIB): $(NETXNGO) $(NETXNGDO) $(ORDER_) $(MAINLIBS) $(NETXNGLIBDEP)
> 55 @$(MAKELIB) $(PLATFORM) $(LD) "$(LDFLAGS)" \
> 56 "$(SOFLAGS)" libNetxNG.$(SOEXT) $@ "$(NETXNGO) $(NETXNGDO)" \
> 57 "$(NETXNGLIBEXTRA)"
> 58
> 59 $(call pcmrule,NETXNG)
> 60 $(noop)
> 61
> 62 $(NETXNGDS): $(NETXNGH) $(NETXNGL) $(XROOTDMAKE) $(ROOTCINTTMPDEP)
> 63 $(MAKEDIR)
> 64 @echo "Generating dictionary $@..."
> 65
> $(ROOTCINTTMP) -f $@ $(call dictModule,NETXNG) -c $(NETXNGINCEXTRA) $(NETXNGH) $(NETXNGL)
> 66
> 67 $(NETXNGMAP): $(RLIBMAP) $(MAKEFILEDEP) $(NETXNGL)
> 68
> $(RLIBMAP) -o $@ -l $(NETXNGLIB) -d $(NETXNGLIBDEPM) -c $(NETXNGL)
> 69
> 70 all-$(MODNAME): $(NETXNGLIB) $(NETXNGMAP)
> 71
> 72 clean-$(MODNAME):
> 73 @rm -f $(NETXNGO) $(NETXNGDO)
> 74
> 75 clean:: clean-$(MODNAME)
> 76
> 77 distclean-$(MODNAME): clean-$(MODNAME)
> 78
> @rm -f $(NETXNGDEP) $(NETXNGDS) $(NETXNGDH) $(NETXNGLIB) $(NETXNGMAP)
> 79
> 80 distclean:: distclean-$(MODNAME)
> 81
> 82 $(NETXNGO) $(NETXNGDO): CXXFLAGS += $(NETXNGINCEXTRA)
> 83 ifneq ($(findstring gnu,$(COMPILER)),)
> 84 # problem in xrootd 3.3.5 headers
> 85 $(NETXNGO) $(NETXNGDO): CXXFLAGS += -Wno-unused-parameter
> 86 endif
>
>
> Added file net/netxng/inc/LinkDef.h
> <http://root.cern.ch/gitweb?p=root.git;f=net/netxng/inc/LinkDef.h;h=cd89ef8;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1 #ifdef __CINT__
> 2
> 3 #pragma link off all globals;
> 4 #pragma link off all classes;
> 5 #pragma link off all functions;
> 6
> 7 #pragma link C++ class TNetXNGFile;
> 8 #pragma link C++ class TNetXNGFileStager;
> 9 #pragma link C++ class TNetXNGSystem;
> 10
> 11 #endif
>
>
> Added file net/netxng/inc/TNetXNGFile.h
> <http://root.cern.ch/gitweb?p=root.git;f=net/netxng/inc/TNetXNGFile.h;h=90cdfca;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1 // @(#)root/netxng:$Id$
> 2
> /*************************************************************************
> 3
> * Copyright (C) 1995-2013, Rene Brun and Fons Rademakers. *
> 4
> * All rights reserved. *
> 5
> * *
> 6
> * For the licensing terms see $ROOTSYS/LICENSE. *
> 7
> * For the list of contributors see $ROOTSYS/README/CREDITS. *
> 8
> *************************************************************************/
> 9
> 10 #ifndef ROOT_TNetXNGFile
> 11 #define ROOT_TNetXNGFile
> 12
> 13
> ////////////////////////////////////////////////////////////////////////////////
> 14
> // //
> 15
> // TNetXNGFile //
> 16
> // //
> 17
> // Authors: Justin Salmon, Lukasz Janyst //
> 18
> // CERN, 2013 //
> 19
> // //
> 20
> // Enables access to XRootD files using the new client. //
> 21
> // //
> 22
> ////////////////////////////////////////////////////////////////////////////////
> 23
> 24 #include "TFile.h"
> 25 #include "TSemaphore.h"
> 26 #ifndef __CINT__
> 27 #include <XrdSys/XrdSysPthread.hh>
> 28 #include <XrdCl/XrdClFileSystem.hh>
> 29 #include <XrdCl/XrdClXRootDResponses.hh>
> 30 #endif
> 31
> 32 namespace XrdCl {
> 33 class File;
> 34 class ResponseHandler;
> 35 }
> 36
> 37 class TNetXNGFile: public TFile {
> 38 private:
> 39 #ifndef __CINT__
> 40 XrdCl::File *fFile; // Underlying XRootD file
> 41 XrdCl::URL *fUrl; // URL of the current file
> 42
> XrdCl::OpenFlags::Flags fMode; // Open mode of the current file
> 43
> XrdSysCondVar *fInitCondVar; // Used to block an async open request
> 44 // if requested
> 45
> Int_t fReadvIorMax; // Max size of a single readv chunk
> 46 Int_t fReadvIovMax; // Max number of readv chunks
> 47 #endif
> 48
> 49 public:
> 50 TNetXNGFile() : TFile(),
> 51
> fFile(0), fUrl(0), fMode(XrdCl::OpenFlags::None), fInitCondVar(0),
> 52 fReadvIorMax(0), fReadvIovMax(0){}
> 53
> TNetXNGFile(const char *url, Option_t *mode = "", const char *title = "",
> 54
> Int_t compress = 1, Int_t netopt = 0, Bool_t parallelopen = kFALSE);
> 55 virtual ~TNetXNGFile();
> 56
> 57 virtual void Init(Bool_t create);
> 58 virtual void Close(const Option_t *option = "");
> 59 virtual void Seek(Long64_t offset, ERelativeTo position = kBeg);
> 60 virtual void SetAsyncOpenStatus(EAsyncOpenStatus status);
> 61 virtual Long64_t GetSize() const;
> 62 virtual Int_t ReOpen(Option_t *modestr);
> 63 virtual Bool_t IsOpen() const;
> 64 virtual Bool_t WriteBuffer(const char *buffer, Int_t length);
> 65 virtual Bool_t ReadBuffer(char *buffer, Int_t length);
> 66
> virtual Bool_t ReadBuffer(char *buffer, Long64_t position, Int_t length);
> 67
> virtual Bool_t ReadBuffers(char *buffer, Long64_t *position, Int_t *length,
> 68 Int_t nbuffs);
> 69
> 70 private:
> 71 virtual Bool_t IsUseable() const;
> 72 virtual Bool_t GetVectorReadLimits();
> 73 virtual void SetEnv();
> 74 #ifndef __CINT__
> 75 XrdCl::OpenFlags::Flags ParseOpenMode(Option_t *modestr);
> 76 #endif
> 77
> 78 TNetXNGFile(const TNetXNGFile &other); // Not implemented
> 79 TNetXNGFile &operator =(const TNetXNGFile &other); // Not implemented
> 80
> 81 ClassDef( TNetXNGFile, 0 ) // ROOT class definition
> 82 };
> 83
> 84 #endif // ROOT_TNetXNGFile
>
>
> Added file net/netxng/inc/TNetXNGFileStager.h
> <http://root.cern.ch/gitweb?p=root.git;f=net/netxng/inc/TNetXNGFileStager.h;h=7a9440c;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1 // @(#)root/netxng:$Id$
> 2
> /*************************************************************************
> 3
> * Copyright (C) 1995-2013, Rene Brun and Fons Rademakers. *
> 4
> * All rights reserved. *
> 5
> * *
> 6
> * For the licensing terms see $ROOTSYS/LICENSE. *
> 7
> * For the list of contributors see $ROOTSYS/README/CREDITS. *
> 8
> *************************************************************************/
> 9
> 10 #ifndef ROOT_TNetXNGFileStager
> 11 #define ROOT_TNetXNGFileStager
> 12
> 13
> ////////////////////////////////////////////////////////////////////////////////
> 14
> // //
> 15
> // TNetXNGFileStager //
> 16
> // //
> 17
> // Authors: Lukasz Janyst, Justin Salmon //
> 18
> // CERN, 2013 //
> 19
> // //
> 20
> // Enables access to XRootD staging capabilities using the new client. //
> 21
> // //
> 22
> ////////////////////////////////////////////////////////////////////////////////
> 23
> 24 #include "TFileStager.h"
> 25
> 26 class TCollection;
> 27 class TNetXNGSystem;
> 28 class TFileCollection;
> 29
> 30 class TNetXNGFileStager: public TFileStager {
> 31
> 32 private:
> 33 TNetXNGSystem *fSystem; // Used to access filesystem interface
> 34
> 35 public:
> 36 TNetXNGFileStager(const char *url = "");
> 37 virtual ~TNetXNGFileStager();
> 38
> 39 Bool_t IsStaged(const char *path);
> 40 Int_t Locate(const char *path, TString &endpath);
> 41
> Int_t LocateCollection(TFileCollection *fc, Bool_t addDummyUrl = kFALSE);
> 42 Bool_t Matches(const char *s);
> 43 Bool_t Stage(const char *path, Option_t *opt = 0);
> 44 Bool_t Stage(TCollection *pathlist, Option_t *opt = 0);
> 45 Bool_t IsValid() const { return (fSystem ? kTRUE : kFALSE); }
> 46
> 47 private:
> 48 UChar_t ParseStagePriority(Option_t *opt);
> 49
> 50 ClassDef( TNetXNGFileStager, 0 ) //! Interface to a 'XRD' staging
> 51 };
> 52
> 53 #endif
>
>
> Added file net/netxng/inc/TNetXNGSystem.h
> <http://root.cern.ch/gitweb?p=root.git;f=net/netxng/inc/TNetXNGSystem.h;h=8184e34;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1 // @(#)root/netxng:$Id$
> 2
> /*************************************************************************
> 3
> * Copyright (C) 1995-2013, Rene Brun and Fons Rademakers. *
> 4
> * All rights reserved. *
> 5
> * *
> 6
> * For the licensing terms see $ROOTSYS/LICENSE. *
> 7
> * For the list of contributors see $ROOTSYS/README/CREDITS. *
> 8
> *************************************************************************/
> 9
> 10 #ifndef ROOT_TNetXNGSystem
> 11 #define ROOT_TNetXNGSystem
> 12
> 13
> ////////////////////////////////////////////////////////////////////////////////
> 14
> // //
> 15
> // TNetXNGSystem //
> 16
> // //
> 17
> // Authors: Justin Salmon, Lukasz Janyst //
> 18
> // CERN, 2013 //
> 19
> // //
> 20
> // Enables access to XRootD filesystem interface using the new client. //
> 21
> // //
> 22
> ////////////////////////////////////////////////////////////////////////////////
> 23
> 24 #include "TSystem.h"
> 25 #include "TCollection.h"
> 26
> 27 #ifndef __CINT__
> 28 #include <XrdCl/XrdClXRootDResponses.hh>
> 29 #include <XrdCl/XrdClURL.hh>
> 30 #endif
> 31
> 32 namespace XrdCl {
> 33 class FileSystem;
> 34 class URL;
> 35 class DirectoryList;
> 36 }
> 37
> 38 class TNetXNGSystem: public TSystem {
> 39
> 40 private:
> 41 #ifndef __CINT__
> 42 private:
> 43 XrdCl::URL *fUrl; // URL of this TSystem
> 44 XrdCl::FileSystem *fFileSystem; // Cached for convenience
> 45
> 46 #endif
> 47
> 48 public:
> 49 TNetXNGSystem(Bool_t owner = kTRUE);
> 50 TNetXNGSystem(const char *url, Bool_t owner = kTRUE);
> 51 virtual ~TNetXNGSystem();
> 52
> 53 virtual void* OpenDirectory(const char* dir);
> 54 virtual Int_t MakeDirectory(const char* dir);
> 55 virtual void FreeDirectory(void *dirp);
> 56 virtual const char* GetDirEntry(void *dirp);
> 57 virtual Int_t GetPathInfo(const char* path, FileStat_t &buf);
> 58 virtual Bool_t ConsistentWith(const char *path, void *dirptr);
> 59 virtual int Unlink(const char *path);
> 60 virtual Bool_t IsPathLocal(const char *path);
> 61 virtual Int_t Locate(const char* path, TString &endurl);
> 62 virtual Int_t Stage(const char* path, UChar_t priority);
> 63 virtual Int_t Stage(TCollection *files, UChar_t priority);
> 64
> 65 ClassDef(TNetXNGSystem, 0 ) // ROOT class definition
> 66 };
> 67
> 68 #endif
>
>
> Added file net/netxng/src/TNetXNGFile.cxx
> <http://root.cern.ch/gitweb?p=root.git;f=net/netxng/src/TNetXNGFile.cxx;h=e99767d;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1 // @(#)root/netxng:$Id$
> 2
> /*************************************************************************
> 3
> * Copyright (C) 1995-2013, Rene Brun and Fons Rademakers. *
> 4
> * All rights reserved. *
> 5
> * *
> 6
> * For the licensing terms see $ROOTSYS/LICENSE. *
> 7
> * For the list of contributors see $ROOTSYS/README/CREDITS. *
> 8
> *************************************************************************/
> 9
> 10
> ////////////////////////////////////////////////////////////////////////////////
> 11
> // //
> 12
> // TNetXNGFile //
> 13
> // //
> 14
> // Authors: Justin Salmon, Lukasz Janyst //
> 15
> // CERN, 2013 //
> 16
> // //
> 17
> // Enables access to XRootD files using the new client. //
> 18
> // //
> 19
> ////////////////////////////////////////////////////////////////////////////////
> 20
> 21 #include "TNetXNGFile.h"
> 22 #include "TEnv.h"
> 23 #include "TSystem.h"
> 24 #include "TTimeStamp.h"
> 25 #include "TVirtualPerfStats.h"
> 26 #include "TVirtualMonitoring.h"
> 27 #include <XrdCl/XrdClURL.hh>
> 28 #include <XrdCl/XrdClFile.hh>
> 29 #include <XrdCl/XrdClXRootDResponses.hh>
> 30 #include <XrdCl/XrdClDefaultEnv.hh>
> 31 #include <iostream>
> 32
> 33
> //------------------------------------------------------------------------------
> 34 // Open handler for async open requests
> 35
> //------------------------------------------------------------------------------
> 36 class TAsyncOpenHandler: public XrdCl::ResponseHandler
> 37 {
> 38 public:
> 39
> //------------------------------------------------------------------------
> 40 // Constructor
> 41
> //------------------------------------------------------------------------
> 42 TAsyncOpenHandler(TNetXNGFile *file)
> 43 {
> 44 fFile = file;
> 45 fFile->SetAsyncOpenStatus(TFile::kAOSInProgress);
> 46 }
> 47
> 48
> //------------------------------------------------------------------------
> 49 // Called when a response to open arrives
> 50
> //------------------------------------------------------------------------
> 51 virtual void HandleResponse(XrdCl::XRootDStatus *status,
> 52 XrdCl::AnyObject *response)
> 53 {
> 54 if (status->IsOK())
> 55 {
> 56 fFile->SetAsyncOpenStatus(TFile::kAOSSuccess);
> 57 }
> 58 else
> 59 {
> 60 fFile->SetAsyncOpenStatus(TFile::kAOSFailure);
> 61 }
> 62
> 63 delete response;
> 64 delete status;
> 65 delete this;
> 66 }
> 67
> 68 private:
> 69 TNetXNGFile *fFile;
> 70 };
> 71
> 72
> //------------------------------------------------------------------------------
> 73 // Async readv handler
> 74
> //------------------------------------------------------------------------------
> 75 class TAsyncReadvHandler: public XrdCl::ResponseHandler
> 76 {
> 77 public:
> 78
> //------------------------------------------------------------------------
> 79 // Constructor
> 80
> //------------------------------------------------------------------------
> 81 TAsyncReadvHandler(std::vector<XrdCl::XRootDStatus*> *statuses,
> 82 Int_t statusIndex,
> 83 TSemaphore *semaphore):
> 84
> fStatuses(statuses), fStatusIndex(statusIndex), fSemaphore(semaphore) {}
> 85
> 86
> 87
> //------------------------------------------------------------------------
> 88 // Handle readv response
> 89
> //------------------------------------------------------------------------
> 90 virtual void HandleResponse(XrdCl::XRootDStatus *status,
> 91 XrdCl::AnyObject *response)
> 92 {
> 93 fStatuses->at(fStatusIndex) = status;
> 94 fSemaphore->Post();
> 95 delete response;
> 96 delete this;
> 97 }
> 98
> 99 private:
> 100
> std::vector<XrdCl::XRootDStatus*> *fStatuses; // Pointer to status vector
> 101
> Int_t fStatusIndex; // Index into status vector
> 102
> TSemaphore *fSemaphore; // Synchronize the responses
> 103 };
> 104
> 105
> 106 ClassImp(TNetXNGFile);
> 107
> 108
> //______________________________________________________________________________
> 109 TNetXNGFile::TNetXNGFile(const char *url,
> 110 Option_t *mode,
> 111 const char *title,
> 112 Int_t compress,
> 113 Int_t /*netopt*/,
> 114 Bool_t parallelopen) :
> 115 TFile(url, "NET", title, compress)
> 116 {
> 117 // Constructor
> 118 //
> 119 // param url: URL of the entry-point server to be contacted
> 120 // param mode: initial file access mode
> 121 // param title: title of the file (shown by ROOT browser)
> 122 // param compress: compression level and algorithm
> 123 // param netopt: TCP window size in bytes (unused)
> 124 // param parallelopen: open asynchronously
> 125
> 126 using namespace XrdCl;
> 127
> 128 fFile = new File();
> 129 fUrl = new URL(std::string(url));
> 130 fInitCondVar = new XrdSysCondVar();
> 131 fUrl->SetProtocol(std::string("root"));
> 132 fMode = ParseOpenMode(mode);
> 133
> 134 // Map ROOT and xrootd environment
> 135 SetEnv();
> 136
> 137 // Init the monitoring system
> 138 if (gMonitoringWriter) {
> 139 if (!fOpenPhases) {
> 140 fOpenPhases = new TList;
> 141 fOpenPhases->SetOwner();
> 142 }
> 143
> gMonitoringWriter->SendFileOpenProgress(this, fOpenPhases, "xrdopen",
> 144 kFALSE);
> 145 }
> 146
> 147 XRootDStatus status;
> 148 if (parallelopen) {
> 149 // Open the file asynchronously
> 150 TAsyncOpenHandler *handler = new TAsyncOpenHandler(this);
> 151
> status = fFile->Open(fUrl->GetURL(), fMode, Access::None, handler);
> 152 if (!status.IsOK()) {
> 153 Error("Open", "%s", status.ToStr().c_str());
> 154 MakeZombie();
> 155 }
> 156 return;
> 157 }
> 158
> 159 // Open the file synchronously
> 160 status = fFile->Open(fUrl->GetURL(), fMode);
> 161 if (!status.IsOK()) {
> 162 Error("Open", "%s", status.ToStr().c_str());
> 163 MakeZombie();
> 164 return;
> 165 }
> 166
> 167 // Initialize the file
> 168 TFile::Init(false);
> 169
> 170 // Get the vector read limits
> 171 GetVectorReadLimits();
> 172 }
> 173
> 174
> //______________________________________________________________________________
> 175 TNetXNGFile::~TNetXNGFile()
> 176 {
> 177 // Destructor
> 178
> 179 if (IsOpen())
> 180 Close();
> 181 delete fFile;
> 182 delete fUrl;
> 183 delete fInitCondVar;
> 184 }
> 185
> 186
> //______________________________________________________________________________
> 187 void TNetXNGFile::Init(Bool_t create)
> 188 {
> 189
> // Initialize the file. Makes sure that the file is really open before
> 190 // calling TFile::Init. It may block.
> 191
> 192 using namespace XrdCl;
> 193
> 194 if (fInitDone) {
> 195 if (gDebug > 1) Info("Init", "TFile::Init already called once");
> 196 return;
> 197 }
> 198
> 199 // If the async open didn't return yet, wait for it
> 200 if (!IsOpen() && fAsyncOpenStatus == kAOSInProgress) {
> 201 fInitCondVar->Wait();
> 202 }
> 203
> 204 // Notify the monitoring system
> 205 if (gMonitoringWriter)
> 206
> gMonitoringWriter->SendFileOpenProgress(this, fOpenPhases, "rootinit",
> 207 kFALSE);
> 208
> 209 // Initialize the file
> 210 TFile::Init(create);
> 211
> 212 // Notify the monitoring system
> 213 if (gMonitoringWriter)
> 214
> gMonitoringWriter->SendFileOpenProgress(this, fOpenPhases, "endopen",
> 215 kTRUE);
> 216
> 217 // Get the vector read limits
> 218 GetVectorReadLimits();
> 219 }
> 220
> 221
> //______________________________________________________________________________
> 222 Long64_t TNetXNGFile::GetSize() const
> 223 {
> 224
> // Get the file size. Returns -1 in the case that the file could not be
> 225 // stat'ed.
> 226
> 227 using namespace XrdCl;
> 228
> 229 // Check the file isn't a zombie or closed
> 230 if (!IsUseable())
> 231 return -1;
> 232
> 233 bool forceStat = true;
> 234 if( fMode == XrdCl::OpenFlags::Read )
> 235 forceStat = false;
> 236
> 237 StatInfo *info = 0;
> 238 if( !fFile->Stat( forceStat, info ).IsOK() )
> 239 return -1;
> 240 Long64_t size = info->GetSize();
> 241 delete info;
> 242 return size;
> 243 }
> 244
> 245
> //______________________________________________________________________________
> 246 Bool_t TNetXNGFile::IsOpen() const
> 247 {
> 248 // Check if the file is open
> 249
> 250 return fFile->IsOpen();
> 251 }
> 252
> 253
> //______________________________________________________________________________
> 254 void TNetXNGFile::SetAsyncOpenStatus(EAsyncOpenStatus status)
> 255 {
> 256 // Set the status of an asynchronous file open
> 257
> 258 fAsyncOpenStatus = status;
> 259 // Unblock Init() if it is waiting
> 260 fInitCondVar->Signal();
> 261 }
> 262
> 263
> //______________________________________________________________________________
> 264 void TNetXNGFile::Close(const Option_t */*option*/)
> 265 {
> 266 // Close the file
> 267 //
> 268
> // param option: if == "R", all TProcessIDs referenced by this file are
> 269 // deleted (is this valid in xrootd context?)
> 270
> 271 TFile::Close();
> 272
> 273 XrdCl::XRootDStatus status = fFile->Close();
> 274 if (!status.IsOK()) {
> 275 Error("Close", "%s", status.ToStr().c_str());
> 276 MakeZombie();
> 277 }
> 278 }
> 279
> 280
> //______________________________________________________________________________
> 281 Int_t TNetXNGFile::ReOpen(Option_t *modestr)
> 282 {
> 283 // Reopen the file with the new access mode
> 284 //
> 285 // param mode: the new access mode
> 286
> // returns: 0 in case the mode was successfully modified, 1 in case
> 287
> // the mode did not change (was already as requested or wrong
> 288
> // input arguments) and -1 in case of failure, in which case
> 289 // the file cannot be used anymore
> 290
> 291 using namespace XrdCl;
> 292 OpenFlags::Flags mode = ParseOpenMode(modestr);
> 293
> 294 // Only Read and Update are valid modes
> 295 if (mode != OpenFlags::Read && mode != OpenFlags::Update) {
> 296
> Error("ReOpen", "mode must be either READ or UPDATE, not %s", modestr);
> 297 return 1;
> 298 }
> 299
> 300 // The mode is not really changing
> 301 if (mode == fMode || (mode == OpenFlags::Update
> 302 && fMode == OpenFlags::New)) {
> 303 return 1;
> 304 }
> 305
> 306 fFile->Close();
> 307 fMode = mode;
> 308
> 309 XRootDStatus st = fFile->Open(fUrl->GetURL(), fMode);
> 310 if (!st.IsOK()) {
> 311 Error("ReOpen", "%s", st.ToStr().c_str());
> 312 return 1;
> 313 }
> 314
> 315 return 0;
> 316 }
> 317
> 318
> //______________________________________________________________________________
> 319 Bool_t TNetXNGFile::ReadBuffer(char *buffer, Int_t length)
> 320 {
> 321 // Read a data chunk of the given size
> 322 //
> 323 // param buffer: a pointer to a buffer big enough to hold the data
> 324 // param length: number of bytes to be read
> 325 // returns: kTRUE in case of failure
> 326
> 327 return ReadBuffer(buffer, fOffset, length);
> 328 }
> 329
> 330
> //______________________________________________________________________________
> 331
> Bool_t TNetXNGFile::ReadBuffer(char *buffer, Long64_t position, Int_t length)
> 332 {
> 333
> // Read a data chunk of the given size, starting from the given offset
> 334 //
> 335 // param buffer: a pointer to a buffer big enough to hold the data
> 336 // param position: offset from the beginning of the file
> 337 // param length: number of bytes to be read
> 338 // returns: kTRUE in case of failure
> 339
> 340 using namespace XrdCl;
> 341 if (gDebug > 0)
> 342 Info("ReadBuffer", "offset: %lld length: %d", position, length);
> 343
> 344 // Check the file isn't a zombie or closed
> 345 if (!IsUseable())
> 346 return kTRUE;
> 347
> 348 // Try to read from cache
> 349 Int_t status;
> 350 if ((status = ReadBufferViaCache(buffer, length))) {
> 351 if (status == 2)
> 352 return kTRUE;
> 353 return kFALSE;
> 354 }
> 355
> 356 // Read the data
> 357 uint32_t bytesRead = 0;
> 358 XRootDStatus st = fFile->Read(position, length, buffer, bytesRead);
> 359 if (gDebug > 0)
> 360
> Info("ReadBuffer", "%s bytes read: %d", st.ToStr().c_str(), bytesRead);
> 361
> 362 if (!st.IsOK()) {
> 363 Error("ReadBuffer", "%s", st.ToStr().c_str());
> 364 return kTRUE;
> 365 }
> 366
> 367 // Bump the globals
> 368 fOffset += length;
> 369 fBytesRead += bytesRead;
> 370 fgBytesRead += bytesRead;
> 371 fReadCalls ++;
> 372 fgReadCalls ++;
> 373
> 374 if (gMonitoringWriter)
> 375 gMonitoringWriter->SendFileReadProgress(this);
> 376
> 377 return kFALSE;
> 378 }
> 379
> 380
> //______________________________________________________________________________
> 381
> Bool_t TNetXNGFile::ReadBuffers(char *buffer, Long64_t *position, Int_t *length,
> 382 Int_t nbuffs)
> 383 {
> 384 // Read scattered data chunks in one operation
> 385 //
> 386
> // param buffer: a pointer to a buffer big enough to hold all of the
> 387 // requested data
> 388
> // param position: position[i] is the seek position of chunk i of len
> 389 // length[i]
> 390 // param length: length[i] is the length of the chunk at offset
> 391 // position[i]
> 392 // param nbuffs: number of chunks
> 393 // returns: kTRUE in case of failure
> 394
> 395 using namespace XrdCl;
> 396
> 397 // Check the file isn't a zombie or closed
> 398 if (!IsUseable())
> 399 return kTRUE;
> 400
> 401 std::vector<ChunkList> chunkLists;
> 402 ChunkList chunks;
> 403 std::vector<XRootDStatus*> *statuses;
> 404 TSemaphore *semaphore;
> 405 Int_t totalBytes = 0;
> 406 Int_t offset = 0;
> 407
> 408 Double_t start = 0;
> 409 if (gPerfStats) start = TTimeStamp();
> 410
> 411 if (fArchiveOffset)
> 412 for (Int_t i = 0; i < nbuffs; i++)
> 413 position[i] += fArchiveOffset;
> 414
> 415 // Build a list of chunks. Put the buffers in the ChunkInfo's
> 416 for (Int_t i = 0; i < nbuffs; ++i) {
> 417 totalBytes += length[i];
> 418
> 419
> // If the length is bigger than max readv size, split into smaller chunks
> 420 if (length[i] > fReadvIorMax) {
> 421 Int_t nsplit = length[i] / fReadvIorMax;
> 422 Int_t rem = length[i] % fReadvIorMax;
> 423 Int_t j;
> 424
> 425 // Add as many max-size chunks as are divisible
> 426 for (j = 0; j < nsplit; ++j) {
> 427 offset = position[i] + (j * fReadvIorMax);
> 428 chunks.push_back(ChunkInfo(offset, fReadvIorMax, buffer));
> 429 }
> 430
> 431 // Add the remainder
> 432 offset = position[i] + (j * fReadvIorMax);
> 433 chunks.push_back(ChunkInfo(offset, rem, buffer));
> 434
> 435 } else {
> 436 offset = position[i];
> 437 chunks.push_back(ChunkInfo(offset, length[i], buffer));
> 438 }
> 439
> 440
> // If there are more than or equal to max chunks, make another chunk list
> 441 if ((Int_t) chunks.size() == fReadvIovMax) {
> 442 chunkLists.push_back(chunks);
> 443 chunks = ChunkList();
> 444 } else if ((Int_t) chunks.size() > fReadvIovMax) {
> 445 chunkLists.push_back(ChunkList(chunks.begin(),
> 446
> chunks.begin() + fReadvIovMax));
> 447
> chunks = ChunkList(chunks.begin() + fReadvIovMax, chunks.end());
> 448 }
> 449 }
> 450
> 451 // Push back the last chunk list
> 452 chunkLists.push_back(chunks);
> 453
> 454 TAsyncReadvHandler *handler;
> 455 XRootDStatus status;
> 456 semaphore = new TSemaphore(0);
> 457 statuses = new std::vector<XRootDStatus*>(chunkLists.size());
> 458
> 459 // Read asynchronously but wait for all responses
> 460 std::vector<ChunkList>::iterator it;
> 461 for (it = chunkLists.begin(); it != chunkLists.end(); ++it)
> 462 {
> 463
> handler = new TAsyncReadvHandler(statuses, it - chunkLists.begin(),
> 464 semaphore);
> 465 status = fFile->VectorRead(*it, buffer, handler);
> 466
> 467 if (!status.IsOK()) {
> 468 Error("ReadBuffers", "%s", status.ToStr().c_str());
> 469 return kTRUE;
> 470 }
> 471 }
> 472
> 473 // Wait for all responses
> 474 for (it = chunkLists.begin(); it != chunkLists.end(); ++it) {
> 475 semaphore->Wait();
> 476 }
> 477
> 478 // Check for errors
> 479 for (it = chunkLists.begin(); it != chunkLists.end(); ++it) {
> 480 XRootDStatus *st = statuses->at(it - chunkLists.begin());
> 481
> 482 if (!st->IsOK()) {
> 483 Error("ReadBuffers", "%s", st->ToStr().c_str());
> 484 delete statuses;
> 485 delete semaphore;
> 486 for( ; it != chunkLists.end(); ++it )
> 487 {
> 488 st = statuses->at( it - chunkLists.begin() );
> 489 delete st;
> 490 }
> 491
> 492 return kTRUE;
> 493 }
> 494 delete st;
> 495 }
> 496
> 497 // Bump the globals
> 498 fBytesRead += totalBytes;
> 499 fgBytesRead += totalBytes;
> 500 fReadCalls ++;
> 501 fgReadCalls ++;
> 502
> 503 if (gPerfStats) {
> 504 fOffset = position[0];
> 505
> gPerfStats->FileReadEvent(this, position[nbuffs - 1] + length[nbuffs - 1]
> 506 - position[0], start);
> 507 }
> 508
> 509 if (gMonitoringWriter)
> 510 gMonitoringWriter->SendFileReadProgress(this);
> 511
> 512 delete statuses;
> 513 delete semaphore;
> 514 return kFALSE;
> 515 }
> 516
> 517
> //______________________________________________________________________________
> 518 Bool_t TNetXNGFile::WriteBuffer(const char *buffer, Int_t length)
> 519 {
> 520 // Write a data chunk
> 521 //
> 522 // param buffer: the data to be written
> 523 // param length: the size of the buffer
> 524 // returns: kTRUE in case of failure
> 525
> 526 using namespace XrdCl;
> 527
> 528 // Check the file isn't a zombie or closed
> 529 if (!IsUseable())
> 530 return kTRUE;
> 531
> 532 // Check the write cache
> 533 Int_t status;
> 534 if ((status = WriteBufferViaCache(buffer, length))) {
> 535 if (status == 2)
> 536 return kTRUE;
> 537 return kFALSE;
> 538 }
> 539
> 540 // Write the data
> 541 XRootDStatus st = fFile->Write(fOffset, length, buffer);
> 542 if (!st.IsOK()) {
> 543 Error("WriteBuffer", "%s", st.ToStr().c_str());
> 544 return kTRUE;
> 545 }
> 546
> 547 // Bump the globals
> 548 fOffset += length;
> 549 fBytesWrite += length;
> 550 fgBytesWrite += length;
> 551
> 552 return kFALSE;
> 553 }
> 554
> 555
> //______________________________________________________________________________
> 556 void TNetXNGFile::Seek(Long64_t offset, ERelativeTo position)
> 557 {
> 558 // Set the position within the file
> 559 //
> 560 // param offset: the new offset relative to position
> 561 // param position: the relative position, either kBeg, kCur or kEnd
> 562
> 563 SetOffset(offset, position);
> 564 }
> 565
> 566
> //______________________________________________________________________________
> 567 XrdCl::OpenFlags::Flags TNetXNGFile::ParseOpenMode(Option_t *modestr)
> 568 {
> 569
> // Parse an file open mode given as a string into an integer that the
> 570 // client can use
> 571 //
> 572 // param option: the file open mode as a string
> 573 // returns: correctly parsed option mode
> 574
> 575 using namespace XrdCl;
> 576 OpenFlags::Flags mode = OpenFlags::None;
> 577 TString mod = ToUpper(TString(modestr));
> 578
> 579 if (mod == "NEW" || mod == "CREATE") mode = OpenFlags::New;
> 580 else if (mod == "RECREATE") mode = OpenFlags::Delete;
> 581 else if (mod == "UPDATE") mode = OpenFlags::Update;
> 582 else if (mod == "READ") mode = OpenFlags::Read;
> 583
> 584 return mode;
> 585 }
> 586
> 587
> //______________________________________________________________________________
> 588 Bool_t TNetXNGFile::IsUseable() const
> 589 {
> 590 // Check the file is open and isn't a zombie
> 591
> 592 if (IsZombie()) {
> 593 Error("TNetXNGFile", "Object is in 'zombie' state");
> 594 return kFALSE;
> 595 }
> 596
> 597 if (!IsOpen()) {
> 598 Error("TNetXNGFile", "The remote file is not open");
> 599 return kFALSE;
> 600 }
> 601
> 602 return kTRUE;
> 603 }
> 604
> 605
> //______________________________________________________________________________
> 606 Bool_t TNetXNGFile::GetVectorReadLimits()
> 607 {
> 608
> // Find the server-specific readv config params. Returns kFALSE in case of
> 609 // error, kTRUE otherwise.
> 610
> 611 using namespace XrdCl;
> 612
> 613 // Check the file isn't a zombie or closed
> 614 if (!IsUseable())
> 615 return kFALSE;
> 616
> 617 URL dataServer(fFile->GetDataServer());
> 618 FileSystem fs(dataServer);
> 619 Buffer arg;
> 620 Buffer *response;
> 621 arg.FromString(std::string("readv_ior_max readv_iov_max"));
> 622
> 623 XRootDStatus status = fs.Query(QueryCode::Config, arg, response);
> 624 if (!status.IsOK()) {
> 625 fReadvIorMax = 2097136;
> 626 fReadvIovMax = 1024;
> 627 return kFALSE;
> 628 }
> 629
> 630 Ssiz_t from = 0;
> 631 TString token;
> 632 fReadvIorMax = fReadvIovMax = 0;
> 633
> 634 while (TString(response->ToString()).Tokenize(token, from, "\n")) {
> 635 if (fReadvIorMax == 0) fReadvIorMax = token.Atoi();
> 636 else if (fReadvIovMax == 0) fReadvIovMax = token.Atoi();
> 637 }
> 638
> 639 delete response;
> 640 return kTRUE;
> 641 }
> 642
> 643
> //______________________________________________________________________________
> 644 void TNetXNGFile::SetEnv()
> 645 {
> 646 // Map ROOT and xrootd environment variables
> 647
> 648 XrdCl::Env *env = XrdCl::DefaultEnv::GetEnv();
> 649 const char *cenv = 0;
> 650 TString val;
> 651
> 652 val = gEnv->GetValue("NetXNG.ConnectionWindow", "");
> 653
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_CONNECTIONWINDOW"))
> 654 || strlen(cenv) <= 0))
> 655 env->PutInt("ConnectionWindow", val.Atoi());
> 656
> 657 val = gEnv->GetValue("NetXNG.ConnectionRetry", "");
> 658
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_CONNECTIONRETRY"))
> 659 || strlen(cenv) <= 0))
> 660 env->PutInt("RequestTimeout", val.Atoi());
> 661
> 662 val = gEnv->GetValue("NetXNG.RequestTimeout", "");
> 663
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_REQUESTTIMEOUT"))
> 664 || strlen(cenv) <= 0))
> 665 env->PutInt("RequestTimeout", val.Atoi());
> 666
> 667 val = gEnv->GetValue("NetXNG.SubStreamsPerChannel", "");
> 668
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_SUBSTREAMSPERCHANNEL"))
> 669 || strlen(cenv) <= 0))
> 670 env->PutInt("SubStreamsPerChannel", val.Atoi());
> 671
> 672 val = gEnv->GetValue("NetXNG.TimeoutResolution", "");
> 673
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_TIMEOUTRESOLUTION"))
> 674 || strlen(cenv) <= 0))
> 675 env->PutInt("TimeoutResolution", val.Atoi());
> 676
> 677 val = gEnv->GetValue("NetXNG.StreamErrorWindow", "");
> 678
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_STREAMERRORWINDOW"))
> 679 || strlen(cenv) <= 0))
> 680 env->PutInt("StreamErrorWindow", val.Atoi());
> 681
> 682 val = gEnv->GetValue("NetXNG.RunForkHandler", "");
> 683
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_RUNFORKHANDLER"))
> 684 || strlen(cenv) <= 0))
> 685 env->PutInt("RunForkHandler", val.Atoi());
> 686
> 687 val = gEnv->GetValue("NetXNG.RedirectLimit", "");
> 688
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_REDIRECTLIMIT"))
> 689 || strlen(cenv) <= 0))
> 690 env->PutInt("RedirectLimit", val.Atoi());
> 691
> 692 val = gEnv->GetValue("NetXNG.WorkerThreads", "");
> 693
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_WORKERTHREADS"))
> 694 || strlen(cenv) <= 0))
> 695 env->PutInt("WorkerThreads", val.Atoi());
> 696
> 697 val = gEnv->GetValue("NetXNG.CPChunkSize", "");
> 698
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_CPCHUNKSIZE"))
> 699 || strlen(cenv) <= 0))
> 700 env->PutInt("CPChunkSize", val.Atoi());
> 701
> 702 val = gEnv->GetValue("NetXNG.CPParallelChunks", "");
> 703
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_CPPARALLELCHUNKS"))
> 704 || strlen(cenv) <= 0))
> 705 env->PutInt("CPParallelChunks", val.Atoi());
> 706
> 707 val = gEnv->GetValue("NetXNG.PollerPreference", "");
> 708
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_POLLERPREFERENCE"))
> 709 || strlen(cenv) <= 0))
> 710 env->PutString("PollerPreference", val.Data());
> 711
> 712 val = gEnv->GetValue("NetXNG.ClientMonitor", "");
> 713
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_CLIENTMONITOR"))
> 714 || strlen(cenv) <= 0))
> 715 env->PutString("ClientMonitor", val.Data());
> 716
> 717 val = gEnv->GetValue("NetXNG.ClientMonitorParam", "");
> 718
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XRD_CLIENTMONITORPARAM"))
> 719 || strlen(cenv) <= 0))
> 720 env->PutString("ClientMonitorParam", val.Data());
> 721
> 722 // Old style netrc file
> 723 TString netrc;
> 724 netrc.Form("%s/.rootnetrc", gSystem->HomeDirectory());
> 725 gSystem->Setenv("XrdSecNETRC", netrc.Data());
> 726
> 727 // For authentication
> 728 val = gEnv->GetValue("XSec.Pwd.ALogFile", "");
> 729 if (val.Length() > 0)
> 730 gSystem->Setenv("XrdSecPWDALOGFILE", val.Data());
> 731
> 732 val = gEnv->GetValue("XSec.Pwd.ServerPuk", "");
> 733 if (val.Length() > 0)
> 734 gSystem->Setenv("XrdSecPWDSRVPUK", val.Data());
> 735
> 736 val = gEnv->GetValue("XSec.GSI.CAdir", "");
> 737 if (val.Length() > 0)
> 738 gSystem->Setenv("XrdSecGSICADIR", val.Data());
> 739
> 740 val = gEnv->GetValue("XSec.GSI.CRLdir", "");
> 741 if (val.Length() > 0)
> 742 gSystem->Setenv("XrdSecGSICRLDIR", val.Data());
> 743
> 744 val = gEnv->GetValue("XSec.GSI.CRLextension", "");
> 745 if (val.Length() > 0)
> 746 gSystem->Setenv("XrdSecGSICRLEXT", val.Data());
> 747
> 748 val = gEnv->GetValue("XSec.GSI.UserCert", "");
> 749 if (val.Length() > 0)
> 750 gSystem->Setenv("XrdSecGSIUSERCERT", val.Data());
> 751
> 752 val = gEnv->GetValue("XSec.GSI.UserKey", "");
> 753 if (val.Length() > 0)
> 754 gSystem->Setenv("XrdSecGSIUSERKEY", val.Data());
> 755
> 756 val = gEnv->GetValue("XSec.GSI.UserProxy", "");
> 757 if (val.Length() > 0)
> 758 gSystem->Setenv("XrdSecGSIUSERPROXY", val.Data());
> 759
> 760 val = gEnv->GetValue("XSec.GSI.ProxyValid", "");
> 761 if (val.Length() > 0)
> 762 gSystem->Setenv("XrdSecGSIPROXYVALID", val.Data());
> 763
> 764 val = gEnv->GetValue("XSec.GSI.ProxyKeyBits", "");
> 765 if (val.Length() > 0)
> 766 gSystem->Setenv("XrdSecGSIPROXYKEYBITS", val.Data());
> 767
> 768 val = gEnv->GetValue("XSec.GSI.ProxyForward", "0");
> 769
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XrdSecGSIPROXYDEPLEN"))
> 770 || strlen(cenv) <= 0))
> 771 gSystem->Setenv("XrdSecGSIPROXYDEPLEN", val.Data());
> 772
> 773 val = gEnv->GetValue("XSec.GSI.CheckCRL", "1");
> 774
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XrdSecGSICRLCHECK"))
> 775 || strlen(cenv) <= 0))
> 776 gSystem->Setenv("XrdSecGSICRLCHECK", val.Data());
> 777
> 778 val = gEnv->GetValue("XSec.GSI.DelegProxy", "0");
> 779
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XrdSecGSIDELEGPROXY"))
> 780 || strlen(cenv) <= 0))
> 781 gSystem->Setenv("XrdSecGSIDELEGPROXY", val.Data());
> 782
> 783 val = gEnv->GetValue("XSec.GSI.SignProxy", "1");
> 784
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XrdSecGSISIGNPROXY"))
> 785 || strlen(cenv) <= 0))
> 786 gSystem->Setenv("XrdSecGSISIGNPROXY", val.Data());
> 787
> 788 val = gEnv->GetValue("XSec.Pwd.AutoLogin", "1");
> 789
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XrdSecPWDAUTOLOG"))
> 790 || strlen(cenv) <= 0))
> 791 gSystem->Setenv("XrdSecPWDAUTOLOG", val.Data());
> 792
> 793 val = gEnv->GetValue("XSec.Pwd.VerifySrv", "1");
> 794
> if (val.Length() > 0 && (!(cenv = gSystem->Getenv("XrdSecPWDVERIFYSRV"))
> 795 || strlen(cenv) <= 0))
> 796 gSystem->Setenv("XrdSecPWDVERIFYSRV", val.Data());
> 797 }
> 798
>
>
> Added file net/netxng/src/TNetXNGFileStager.cxx
> <http://root.cern.ch/gitweb?p=root.git;f=net/netxng/src/TNetXNGFileStager.cxx;h=6584f14;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1 // @(#)root/netx:$Id$
> 2
> /*************************************************************************
> 3
> * Copyright (C) 1995-2004, Rene Brun and Fons Rademakers. *
> 4
> * All rights reserved. *
> 5
> * *
> 6
> * For the licensing terms see $ROOTSYS/LICENSE. *
> 7
> * For the list of contributors see $ROOTSYS/README/CREDITS. *
> 8
> *************************************************************************/
> 9
> 10
> ////////////////////////////////////////////////////////////////////////////////
> 11
> // //
> 12
> // TNetXNGFileStager //
> 13
> // //
> 14
> // Authors: Justin Salmon, Lukasz Janyst //
> 15
> // CERN, 2013 //
> 16
> // //
> 17
> // Enables access to XRootD staging capabilities using the new client. //
> 18
> // //
> 19
> ////////////////////////////////////////////////////////////////////////////////
> 20
> 21 #include "TNetXNGFileStager.h"
> 22 #include "TNetXNGSystem.h"
> 23 #include "THashList.h"
> 24 #include "TFileInfo.h"
> 25 #include "TFileCollection.h"
> 26 #include <XrdCl/XrdClFileSystem.hh>
> 27
> 28 ClassImp( TNetXNGFileStager);
> 29
> 30
> //______________________________________________________________________________
> 31 TNetXNGFileStager::TNetXNGFileStager(const char *url) :
> 32 TFileStager("xrd")
> 33 {
> 34 // Constructor
> 35 //
> 36 // param url: the URL of the entry-point server
> 37
> 38 fSystem = new TNetXNGSystem(url);
> 39 }
> 40
> 41
> //______________________________________________________________________________
> 42 TNetXNGFileStager::~TNetXNGFileStager()
> 43 {
> 44 // Destructor
> 45
> 46 delete fSystem;
> 47 }
> 48
> 49
> //______________________________________________________________________________
> 50 Bool_t TNetXNGFileStager::IsStaged(const char *path)
> 51 {
> 52 // Check if a file is staged
> 53 //
> 54 // param path: the URL of the file
> 55
> 56 FileStat_t st;
> 57 if (fSystem->GetPathInfo(path, st) != 0) {
> 58 if (gDebug > 0)
> 59 Info("IsStaged", "path %s cannot be stat'ed", path);
> 60 return kFALSE;
> 61 }
> 62
> 63 if (R_ISOFF(st.fMode)) {
> 64 if (gDebug > 0)
> 65 Info("IsStaged", "path '%s' is offline", path);
> 66 return kFALSE;
> 67 }
> 68
> 69 return kTRUE;
> 70 }
> 71
> 72
> //______________________________________________________________________________
> 73 Int_t TNetXNGFileStager::Locate(const char *path, TString &url)
> 74 {
> 75 // Get actual endpoint URL
> 76 //
> 77 // param path: the entry-point URL
> 78 // param endpath: the actual endpoint URL
> 79
> // returns: 0 in the case of success and 1 if any error occurred
> 80
> 81 return fSystem->Locate(path, url);
> 82 }
> 83
> 84
> //______________________________________________________________________________
> 85 Int_t TNetXNGFileStager::LocateCollection(TFileCollection *fc,
> 86 Bool_t addDummyUrl)
> 87 {
> 88 // Bulk locate request for a collection of files
> 89 //
> 90 // param fc: collection of files to be located
> 91
> // param addDummyUrl: append a dummy noop URL if the file is not staged or
> 92 // redirector == endpoint
> 93
> // returns: < 0 in case of errors, number of files processed
> 94 // otherwise
> 95
> 96 if (!fc) {
> 97 Error("LocateCollection", "No input collection given");
> 98 return -1;
> 99 }
> 100
> 101 int numFiles = 0;
> 102 TFileInfo *info;
> 103 TIter it(fc->GetList());
> 104 TString startUrl, endUrl;
> 105
> 106 while ((info = dynamic_cast<TFileInfo *>(it.Next())) != NULL) {
> 107 startUrl = info->GetCurrentUrl()->GetUrl();
> 108
> 109 // File not staged
> 110 if (fSystem->Locate(startUrl.Data(), endUrl)) {
> 111 info->ResetBit(TFileInfo::kStaged);
> 112
> 113 if (addDummyUrl)
> 114 info->AddUrl("noop://none", kTRUE);
> 115
> 116 if (gDebug > 1)
> 117 Info("LocateCollection", "Not found: %s", startUrl.Data());
> 118 }
> 119
> 120 // File staged
> 121 else {
> 122 info->SetBit(TFileInfo::kStaged);
> 123
> 124 if (startUrl != endUrl) {
> 125 info->AddUrl(endUrl.Data(), kTRUE);
> 126 } else if (addDummyUrl) {
> 127 // Returned URL identical to redirector URL
> 128 info->AddUrl("noop://redir", kTRUE);
> 129 }
> 130
> 131 if (gDebug > 1)
> 132
> Info("LocateCollection", "Found: %s --> %s", startUrl.Data(),
> 133 endUrl.Data());
> 134 }
> 135 numFiles++;
> 136 }
> 137
> 138 return numFiles;
> 139 }
> 140
> 141
> //______________________________________________________________________________
> 142 Bool_t TNetXNGFileStager::Matches(const char *s)
> 143 {
> 144
> // Returns kTRUE if stager 's' is compatible with current stager. Avoids
> 145
> // multiple instantiations of the potentially the same TNetXNGFileStager.
> 146
> 147 return ((s && (fName == s)) ? kTRUE : kFALSE);
> 148 }
> 149
> 150
> //______________________________________________________________________________
> 151 Bool_t TNetXNGFileStager::Stage(const char *path, Option_t *opt)
> 152 {
> 153 // Issue a stage request for a single file
> 154 //
> 155 // param path: the path of the file to stage
> 156
> // param opt: defines 'option' and 'priority' for 'Prepare': the format is
> 157 // opt = "option=o priority=p"
> 158
> 159 Int_t priority = ParseStagePriority(opt);
> 160 return fSystem->Stage(path, priority);
> 161 }
> 162
> 163
> //______________________________________________________________________________
> 164 Bool_t TNetXNGFileStager::Stage(TCollection *paths, Option_t *opt)
> 165 {
> 166 // Issue stage requests for multiple files
> 167 //
> 168 // param pathlist: list of paths of files to stage
> 169
> // param opt: defines 'option' and 'priority' for 'Prepare': the
> 170 // format is opt = "option=o priority=p"
> 171
> 172 Int_t priority = ParseStagePriority(opt);
> 173 return fSystem->Stage(paths, priority);
> 174 }
> 175
> 176
> //______________________________________________________________________________
> 177 UChar_t TNetXNGFileStager::ParseStagePriority(Option_t *opt)
> 178 {
> 179 // Get a staging priority value from an option string
> 180
> 181 UChar_t priority = 0;
> 182 Ssiz_t from = 0;
> 183 TString token;
> 184
> 185 while (TString(opt).Tokenize(token, from, "[ ,|]")) {
> 186 if (token.Contains("priority=")) {
> 187 token.ReplaceAll("priority=", "");
> 188 if (token.IsDigit()) {
> 189 priority = token.Atoi();
> 190 }
> 191 }
> 192 }
> 193
> 194 return priority;
> 195 }
>
>
> Added file net/netxng/src/TNetXNGSystem.cxx
> <http://root.cern.ch/gitweb?p=root.git;f=net/netxng/src/TNetXNGSystem.cxx;h=0950f1b;hb=fc05b7c99b98d917157ca777884ee97c0237a70a>
>
>
> 1 // @(#)root/netx:$Id$
> 2
> /*************************************************************************
> 3
> * Copyright (C) 1995-2004, Rene Brun and Fons Rademakers. *
> 4
> * All rights reserved. *
> 5
> * *
> 6
> * For the licensing terms see $ROOTSYS/LICENSE. *
> 7
> * For the list of contributors see $ROOTSYS/README/CREDITS. *
> 8
> *************************************************************************/
> 9
> 10
> ////////////////////////////////////////////////////////////////////////////////
> 11
> // //
> 12
> // TNetXNGSystem //
> 13
> // //
> 14
> // Authors: Justin Salmon, Lukasz Janyst //
> 15
> // CERN, 2013 //
> 16
> // //
> 17
> // Enables access to XRootD filesystem interface using the new client. //
> 18
> // //
> 19
> ////////////////////////////////////////////////////////////////////////////////
> 20
> 21 #include "TNetXNGSystem.h"
> 22 #include "TFileStager.h"
> 23 #include "Rtypes.h"
> 24 #include "TList.h"
> 25 #include "TUrl.h"
> 26 #include <XrdCl/XrdClFileSystem.hh>
> 27 #include <XrdCl/XrdClXRootDResponses.hh>
> 28
> 29 ClassImp(TNetXNGSystem);
> 30
> 31 namespace
> 32 {
> 33 struct DirectoryInfo {
> 34
> XrdCl::URL *fUrl; // Path of this directory
> 35 XrdCl::DirectoryList *fDirList; // Directory listing
> 36
> XrdCl::DirectoryList::Iterator *fDirListIter; // Iterator for this listing
> 37
> 38 public:
> 39
> DirectoryInfo(const char *dir) : fUrl(new XrdCl::URL(dir)), fDirList(0), fDirListIter(0) {}
> 40
> 41 ~DirectoryInfo() {
> 42 delete fUrl;
> 43 delete fDirList;
> 44 }
> 45 };
> 46 }
> 47
> 48
> //______________________________________________________________________________
> 49 TNetXNGSystem::TNetXNGSystem(Bool_t /*owner*/) :
> 50 TSystem("-root", "Net file Helper System"), fUrl(0), fFileSystem(0)
> 51 {
> 52 // Constructor: Create system class without connecting to server
> 53 //
> 54 // param owner: (unused)
> 55
> 56
> // Name must start with '-' to bypass the TSystem singleton check, then
> 57 // be changed to "root"
> 58 SetName("root");
> 59 }
> 60
> 61
> //______________________________________________________________________________
> 62 TNetXNGSystem::TNetXNGSystem(const char *url, Bool_t /*owner*/) :
> 63 TSystem("-root", "Net file Helper System")
> 64 {
> 65 // Constructor: Create system class and connect to server
> 66 //
> 67 // param url: URL of the entry-point server to be contacted
> 68 // param owner: (unused)
> 69
> 70 using namespace XrdCl;
> 71
> 72 // Name must start with '-' to bypass the TSystem singleton check
> 73 SetName("root");
> 74 fUrl = new URL(std::string(url));
> 75 fFileSystem = new FileSystem(fUrl->GetURL());
> 76 }
> 77
> 78
> //______________________________________________________________________________
> 79 TNetXNGSystem::~TNetXNGSystem()
> 80 {
> 81 // Destructor
> 82
> 83 delete fFileSystem;
> 84 delete fUrl;
> 85 }
> 86
> 87
> //______________________________________________________________________________
> 88 void* TNetXNGSystem::OpenDirectory(const char *dir)
> 89 {
> 90 // Open a directory
> 91 //
> 92 // param dir: the name of the directory to open
> 93 // returns: a non-zero pointer (with no special purpose) in case of
> 94 // success, 0 in case of error
> 95
> 96 using namespace XrdCl;
> 97
> 98 DirectoryInfo *dirInfo = new DirectoryInfo(dir);
> 99 return (void *) dirInfo;
> 100 }
> 101
> 102
> //______________________________________________________________________________
> 103 Int_t TNetXNGSystem::MakeDirectory(const char *dir)
> 104 {
> 105 // Create a directory
> 106 //
> 107 // param dir: the directory name
> 108 // returns: 0 on success, -1 otherwise
> 109
> 110 using namespace XrdCl;
> 111 URL url(dir);
> 112
> XRootDStatus st = fFileSystem->MkDir(url.GetPath(), MkDirFlags::MakePath,
> 113 Access::None);
> 114 if (!st.IsOK()) {
> 115 Error("MakeDirectory", "%s", st.GetErrorMessage().c_str());
> 116 return -1;
> 117 }
> 118
> 119 return 0;
> 120 }
> 121
> 122
> //______________________________________________________________________________----
> 123 void TNetXNGSystem::FreeDirectory(void *dirp)
> 124 {
> 125 // Free a directory
> 126 //
> 127 // param dirp: the pointer to the directory to be freed
> 128
> 129 delete (DirectoryInfo *) dirp;
> 130 }
> 131
> 132
> //______________________________________________________________________________
> 133 const char* TNetXNGSystem::GetDirEntry(void *dirp)
> 134 {
> 135 // Get a directory entry.
> 136 //
> 137 // param dirp: the directory pointer
> 138 // returns: 0 in case there are no more entries
> 139
> 140 using namespace XrdCl;
> 141 DirectoryInfo *dirInfo = (DirectoryInfo *) dirp;
> 142
> 143 if (!dirInfo->fDirList) {
> 144 XRootDStatus st = fFileSystem->DirList(dirInfo->fUrl->GetPath(),
> 145 DirListFlags::Locate,
> 146 dirInfo->fDirList);
> 147 if (!st.IsOK()) {
> 148 Error("GetDirEntry", "%s", st.GetErrorMessage().c_str());
> 149 return 0;
> 150 }
> 151
> dirInfo->fDirListIter = new DirectoryList::Iterator(dirInfo->fDirList->Begin());
> 152 }
> 153
> 154 if (*(dirInfo->fDirListIter) != dirInfo->fDirList->End()) {
> 155
> const char *filename = (**(dirInfo->fDirListIter))->GetName().c_str();
> 156 (*(dirInfo->fDirListIter))++;
> 157 return filename;
> 158
> 159 } else {
> 160 return 0;
> 161 }
> 162 }
> 163
> 164
> //______________________________________________________________________________
> 165 Int_t TNetXNGSystem::GetPathInfo(const char *path, FileStat_t &buf)
> 166 {
> 167 // Get info about a file (stat)
> 168 //
> 169 // param path: the path of the file to stat (in)
> 170 // param buf: structure that will hold the stat info (out)
> 171 // returns: 0 if success, 1 if the file could not be stat'ed
> 172
> 173 using namespace XrdCl;
> 174 StatInfo *info = 0;
> 175 URL target(path);
> 176 XRootDStatus st = fFileSystem->Stat(target.GetPath(), info);
> 177
> 178 if (!st.IsOK()) {
> 179
> 180 Error("GetPathInfo", "Error: %s", st.GetErrorMessage().c_str());
> 181 delete info;
> 182 return 1;
> 183
> 184 } else {
> 185
> 186 // Flag offline files
> 187 if (info->TestFlags(StatInfo::Offline)) {
> 188 buf.fMode = kS_IFOFF;
> 189 } else {
> 190 std::stringstream sstr(info->GetId());
> 191 Long64_t id;
> 192 sstr >> id;
> 193
> 194 buf.fDev = (id >> 32);
> 195 buf.fIno = (id & 0x00000000FFFFFFFF);
> 196 buf.fUid = -1; // not available
> 197 buf.fGid = -1; // not available
> 198 buf.fIsLink = 0; // not available
> 199 buf.fSize = info->GetSize();
> 200 buf.fMtime = info->GetModTime();
> 201
> 202 if (info->TestFlags(StatInfo::XBitSet))
> 203 buf.fMode = (kS_IFREG | kS_IXUSR | kS_IXGRP | kS_IXOTH);
> 204
> if (info->GetFlags() == 0) buf.fMode = kS_IFREG;
> 205
> if (info->TestFlags(StatInfo::IsDir)) buf.fMode = kS_IFDIR;
> 206
> if (info->TestFlags(StatInfo::Other)) buf.fMode = kS_IFSOCK;
> 207
> if (info->TestFlags(StatInfo::IsReadable)) buf.fMode |= kS_IRUSR;
> 208
> if (info->TestFlags(StatInfo::IsWritable)) buf.fMode |= kS_IWUSR;
> 209 }
> 210 }
> 211
> 212 delete info;
> 213 return 0;
> 214 }
> 215
> 216
> //______________________________________________________________________________
> 217 Bool_t TNetXNGSystem::ConsistentWith(const char *path, void *dirptr)
> 218 {
> 219
> // Check consistency of this helper with the one required by 'path' or
> 220 // 'dirptr'
> 221 //
> 222 // param path: the path to check
> 223 // param dirptr: the directory pointer to check
> 224
> 225 using namespace XrdCl;
> 226
> 227
> // Standard check: only the protocol part of 'path' is required to match
> 228 Bool_t checkstd = TSystem::ConsistentWith(path, dirptr);
> 229 if (!checkstd)
> 230 return kFALSE;
> 231
> 232 URL url(path);
> 233 Bool_t checknet = path ? kFALSE : kTRUE;
> 234
> 235 if (gDebug > 1)
> 236
> Info("ConsistentWith", "fUser:'%s' (%s), fHost:'%s' (%s), fPort:%d (%d)",
> 237 fUrl->GetUserName().c_str(), url.GetUserName().c_str(),
> 238 fUrl->GetHostName().c_str(), url.GetHostName().c_str(),
> 239 fUrl->GetPort(), url.GetPort());
> 240
> 241 // Require match of 'user' and 'host'
> 242 if (fUrl->GetUserName() == url.GetUserName()
> 243 && fUrl->GetHostName() == url.GetHostName()
> 244 && fUrl->GetPort() == url.GetPort())
> 245 checknet = kTRUE;
> 246
> 247 return (checkstd && checknet);
> 248 }
> 249
> 250
> //______________________________________________________________________________
> 251 int TNetXNGSystem::Unlink(const char *path)
> 252 {
> 253 // Unlink a file on the remote server
> 254 //
> 255 // param path: the path of the file to unlink
> 256 // returns: 0 on success, -1 otherwise
> 257
> 258 using namespace XrdCl;
> 259 StatInfo *info;
> 260 URL url(path);
> 261
> 262 // Stat the path to find out if it's a file or a directory
> 263 XRootDStatus st = fFileSystem->Stat(url.GetPath(), info);
> 264 if (!st.IsOK()) {
> 265 Error("Unlink", "%s", st.GetErrorMessage().c_str());
> 266 delete info;
> 267 return -1;
> 268 }
> 269
> 270 if (info->TestFlags(StatInfo::IsDir))
> 271 st = fFileSystem->RmDir(url.GetPath());
> 272 else
> 273 st = fFileSystem->Rm(url.GetPath());
> 274 delete info;
> 275
> 276 if (!st.IsOK()) {
> 277 Error("Unlink", "%s", st.GetErrorMessage().c_str());
> 278 return -1;
> 279 }
> 280
> 281 return 0;
> 282 }
> 283
> 284
> //______________________________________________________________________________
> 285 Bool_t TNetXNGSystem::IsPathLocal(const char *path)
> 286 {
> 287 // Is this path a local path?
> 288 //
> 289 // param path: the URL of the path to check
> 290 // returns: kTRUE if the path is local, kFALSE otherwise
> 291
> 292 return TSystem::IsPathLocal(path);
> 293 }
> 294
> 295
> //______________________________________________________________________________
> 296 Int_t TNetXNGSystem::Locate(const char *path, TString &endurl)
> 297 {
> 298 // Get the endpoint URL of a file.
> 299 //
> 300 // param path: the entry-point URL of the file (in)
> 301 // param endurl: the endpoint URL of the file (out)
> 302 // returns: 0 in case of success and 1 if the file could not be
> 303 // stat'ed.
> 304
> 305 using namespace XrdCl;
> 306 LocationInfo *info = 0;
> 307 URL pathUrl(path);
> 308
> 309 // Locate the file
> 310
> XRootDStatus st = fFileSystem->Locate(pathUrl.GetPath(), OpenFlags::None,
> 311 info);
> 312 if (!st.IsOK()) {
> 313 Error("Locate", "%s", st.GetErrorMessage().c_str());
> 314 delete info;
> 315 return 1;
> 316 }
> 317
> 318 // Return the first address
> 319 endurl = info->Begin()->GetAddress();
> 320 delete info;
> 321 return 0;
> 322 }
> 323
> 324
> //______________________________________________________________________________
> 325 Int_t TNetXNGSystem::Stage(const char* path, UChar_t priority)
> 326 {
> 327 // Issue a stage request for a single file
> 328 //
> 329 // param path: the path of the file to stage
> 330
> // param opt: defines 'option' and 'priority' for 'Prepare': the format is
> 331 // opt = "option=o priority=p"
> 332 // returns: 0 for success, -1 for error
> 333
> 334 TList *files = new TList();
> 335 files->Add((TObject *) new TUrl(path));
> 336 return Stage((TCollection *) files, priority);
> 337 }
> 338
> 339
> //______________________________________________________________________________
> 340 Int_t TNetXNGSystem::Stage(TCollection *files, UChar_t priority)
> 341 {
> 342 // Issue stage requests for multiple files
> 343 //
> 344 // param pathlist: list of paths of files to stage
> 345
> // param opt: defines 'option' and 'priority' for 'Prepare': the
> 346 // format is opt = "option=o priority=p"
> 347 // returns: 0 for success, -1 for error
> 348
> 349 using namespace XrdCl;
> 350 std::vector<std::string> fileList;
> 351 TIter it(files);
> 352 TObject *object = 0;
> 353
> 354 while ((object = (TObject *) it.Next())) {
> 355
> 356 TString path = TFileStager::GetPathName(object);
> 357 if (path == "") {
> 358 Warning("Stage", "object is of unexpected type %s - ignoring",
> 359 object->ClassName());
> 360 continue;
> 361 }
> 362
> 363 fileList.push_back(std::string(URL(path.Data()).GetPath()));
> 364 }
> 365
> 366 Buffer *response;
> 367
> XRootDStatus st = fFileSystem->Prepare(fileList, PrepareFlags::Stage,
> 368
> (uint8_t) priority, response);
> 369 if (!st.IsOK()) {
> 370 Error("Stage", "%s", st.GetErrorMessage().c_str());
> 371 return -1;
> 372 }
> 373
> 374 return 0;
> 375 }
> 376
>
>
>
--
Org: CERN, European Laboratory for Particle Physics.
Mail: 1211 Geneve 23, Switzerland
E-Mail: [log in to unmask] Phone: +41 22 7679248
WWW: http://fons.rademakers.org Fax: +41 22 7669640
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|
