 |
 |
Importing the XRootD.client after ROOT and opening a file causes XRootD to deadlock when the python exits:
import ROOT
import XRootD.client
f = ROOT.TFile.Open("root://eospublic.cern.ch//eos/opendata/lhcb/AntimatterMatters2017/data/B2HHH_MagnetDown.root")
tree = f.Get("DecayTree")The cause is the XRootD atexit handler stopping the job manager:
Meanwhile ROOT's atexit handler which [deletes all objects and implicitly closes any open files]9https://github.com/root-project/root/blob/8844e28d7792900932b826c37f755504cb75067f/bindings/pyroot/pythonizations/python/ROOT/__init__.py#L106).
This means that if ROOT is imported before XRootD.client the ROOT cleanup will run after the job manager has been destroyed and the kXR_close message will never be picked up (atexit handlers are ran in the reverse registeration order):
[2021-11-08 10:08:53.333384 +0100][Debug ][JobMgr ] Stopping the job manager...
[2021-11-08 10:08:53.333414 +0100][Dump ][JobMgr ] Stopping worker #0...
[2021-11-08 10:08:53.333619 +0100][Dump ][JobMgr ] Worker #0 stopped
[2021-11-08 10:08:53.333631 +0100][Dump ][JobMgr ] Stopping worker #1...
[2021-11-08 10:08:53.333784 +0100][Dump ][JobMgr ] Worker #1 stopped
[2021-11-08 10:08:53.333796 +0100][Dump ][JobMgr ] Stopping worker #2...
[2021-11-08 10:08:53.333875 +0100][Dump ][JobMgr ] Worker #2 stopped
[2021-11-08 10:08:53.333880 +0100][Debug ][JobMgr ] Job manager stopped
[2021-11-08 10:08:53.333893 +0100][Debug ][TaskMgr ] Stopping the task manager...
[2021-11-08 10:08:53.333977 +0100][Debug ][TaskMgr ] Task manager stopped
[2021-11-08 10:08:53.333982 +0100][Debug ][Poller ] Stopping the poller...
[2021-11-08 10:08:53.334189 +0100][Debug ][File ] [0x7a2ebca0@root://eospublic.cern.ch:1094//eos/opendata/lhcb/AntimatterMatters2017/data/B2HHH_MagnetDown.root?xrdcl.requuid=a4583384-30b5-4e69-98a0-792e29621a65] Sending a close command for handle 0x0 to st-096-ff761117.cern.ch:1095
[2021-11-08 10:08:53.334206 +0100][Dump ][XRootD ] [st-096-ff761117.cern.ch:1095] Sending message kXR_close (handle: 0x00000000)
[2021-11-08 10:08:53.334216 +0100][Debug ][ExDbgMsg ] [st-096-ff761117.cern.ch:1095] MsgHandler created: 0x7ae787e0 (message: kXR_close (handle: 0x00000000) ).
[2021-11-08 10:08:53.334236 +0100][Dump ][PostMaster ] [st-096-ff761117.cern.ch:1095] Sending message kXR_close (handle: 0x00000000) (0x7ae61610) through substream 0 expecting answer at 0
# The process is now deadlocked:
To see this more easily you can run this in an interactive (i)python REPL:
# Add printouts each time a atexit handler is registered
import atexit
orig_register = atexit.register
def my_register(func, *args, **kwargs):
print("Registering atexit handler from", getattr(func, "__module__", "Unknown"), ":", func, args, kwargs)
return orig_register(func, *args, **kwargs)
atexit.register = my_register
# Open a ROOT file but don't clos it
import ROOT
import XRootD.client
f = ROOT.TFile.Open("root://eospublic.cern.ch//eos/opendata/lhcb/AntimatterMatters2017/data/B2HHH_MagnetDown.root")
tree = f.Get("DecayTree")
# Calling f.Close() here would prevent the deadlock
# Run the relvant parts of the atexit handlers by hand to make the issue clearer
from pyxrootd import client
import libROOTPythonizations
# Would be ran as part of XRootD.client's exit handler
client.__XrdCl_Stop_Threads()
# Would be ran as part of ROOT's exit handler
# This line will deadlock
libROOTPythonizations.ClearProxiedObjects()—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1