 |
 |
Hi Andy, As the person in charge of the CMS framework, I can say that at least for CMS (don't know about the other experiments) we call 'close' on all files opened by ROOT before doing the forking (that is how we made our code work via local file access). As Brian mentioned, it would also be fine if we had an API to call before (or after) doing all the 'close' calls which told xrootd that once we've closed all files it should also release any other 'external' resources held by xrootd. That way it becomes the client's responsibility to tell xrootd to free both the files and any extra external resources. Just wanted to be sure that was clear, Chris Christopher Jones Fermi National Accelerator Laboratory [log in to unmask] On Nov 22, 2010, at 1:35 PM, Andrew Hanushevsky wrote: > Unfortunately, there is no good portable way of doing this. The way it's handled in the server is to set the "close on exec" flag for any file descriptor obtained. That's not ideal because you still have an opening to leak a descriptor. Linux provides a special flag to do this for *every* call that obtains a file descriptor (pretty ugly but I understand why they did it that way). Even if we did a complete scan through the code to make sure that file descriptors are not likely to leak, that doesn't address what an application or plug-in might do (we can only work on hope here). > I'll look into comming up with a standard mechanism to deal with this that is portable. Until then, I suspect, it's hunt and peck to catch all of the ones that are there now. > > Andy > > P.S. atfork() is relatively useless since you don't know what outstanding descriptors should really be closed at that point. > > On Mon, 22 Nov 2010, Lukasz Janyst wrote: > >> On Mon, Nov 22, 2010 at 4:00 PM, Brian Bockelman <[log in to unmask]> wrote: >>> What is the fork safety of XrdClient? We've recently found issues where if we fork() a child process in CMSSW after open/closing a file, that some protocols which persist connections become confused when there are now two processes using the same TCP connection. >>> >>> We would prefer XrdClient to handle the issue transparently if possible. If this is not possible (and we'd strongly prefer the former), we'd like a mechanism to release all connection resource that we just call once prior to forking. We would *not* have to clean this up at a per-file basis as we have no knowledge which files are connected to which server. >>> >>> At any rate, before we get too far ahead of ourselves... what's available? >> >> Hi Brian, >> >> I don't think there is any safety mechanism for that at the moment. >> There is the pthread_atfork function that let's you register a couple >> of callbacks to be called before and after forking, so we could, in >> principle handle forking transparently. This function, however, is not >> really a standard one, so it may behave differently on different >> (Linux) platforms or may not be present at all. I'll have a closer >> look and let you know. In any case, providing a release mechanism >> doesn't look like a big development. >> >> Cheers, >> Lukasz >>