 |
 |
Hi Andy, Thanks ... I reinspected the XrdClFile::Close() path and now I see that having a delayed destruction of XrdPosixFile / XrdOucCacheIO is really the only way to cleanup things properly. Matevz On 03/17/14 22:17, Andrew Hanushevsky wrote: > Hi Matevz, > > That doesn't seem, at this point, it will buy you much. If you set a flag saying > "throw away" and close the file anyway, it doesn't get closed. It's not clear if > you can ever close it afterwards. Even if you can, someone has to do it as you > will leak memory unless it is successfully closed. > > Andy > > On Mon, 17 Mar 2014, Matevz Tadel wrote: > >> Hi Andy, >> >> Yes, we want to throw it away :) We have a mini-test with async read + condition >> variable running so we'll know soon if there's any issue implanting this into >> the proxy code. >> >> Matevz >> >> >> >> On 03/17/14 19:00, Andrew Hanushevsky wrote: >>> Hi Matevz, >>> >>> The answer is driven by how important is the pending data? If it's important >>> then it should be allowed to be added to the cache set even when the file is >>> closed. If it's not important then it can be thrown away with impunity. Throwing >>> away data here is far easier than trying to keep it. You can mark your prefetch >>> object as dead and safely delete the file object (the prefetch object would have >>> to delete itself eventually after it threw away the last piece of data, I >>> assume). >>> >>> Otherwise, yes, we would need to add a "Delete()" method that could delay the >>> actual destruction of the object. Note that the object would need to remove >>> itself from the file table but otherwise delay freeing memory. Answers your >>> question? >>> >>> Andy >>> >>> -----Original Message----- From: Matevz Tadel >>> Sent: Monday, March 17, 2014 6:33 PM >>> To: Andrew Hanushevsky ; xrootd-dev >>> Subject: Re: XrdCl interrupting sync read >>> >>> Hi Andy, >>> >>> OK, I see ... then we're back to the discussion you and Alja had a week ago >>> about XrdPosixFile destruction in caching proxy :) >>> >>> We can try to bypass this problem with async reads (and tell ResponseHandler >>> object to discard the result and auto-destruct when) or you can help us to delay >>> the destruction. Or we try both. What do you think is better? >>> >>> What happens now is that when we shut down X-hundred clients simultaneously, >>> some of them take quite a long while to die -- we've already shortened this >>> significantly by removing in-memory blocks from the write queue but there are >>> horrible outliers in prefetch-read, too (30s) -- the problem is that many >>> prefetching threads need to be notified at the same time that they should stop >>> -- and they aren't so "prefetching" goes on for a long while. >>> >>> Matevz >>> >>> On 3/17/14 3:37 PM, Andrew Hanushevsky wrote: >>>> Hi Matevz, >>>> >>>> The current way the new client works is that close() returns an error if there >>>> are any outstanding operations. The way I get around that is to keep track of >>>> the current outstanding operations and when I need to "get rid" of the file, >>>> the >>>> higher level "close()" returns a success and the object is placed in the >>>> background and the real close happens when all of the outstanding operations >>>> complete. I don't know of any way to cancel an outstanding operation. >>>> >>>> Andy >>>> >>>> -----Original Message----- From: Matevz Tadel >>>> Sent: Monday, March 17, 2014 2:37 PM >>>> To: Andrew Hanushevsky ; xrootd-dev >>>> Subject: Re: XrdCl interrupting sync read >>>> >>>> Hi Andy, >>>> >>>> On 03/17/14 14:26, Andrew Hanushevsky wrote: >>>>> Hi Matevz, >>>>> >>>>> All of xrootd avoids signals like the plague . The only exceptions is >>>>> server-side async I/O where it's required for portability. Then again, of >>>>> late, >>>>> everyone turns that capability off. Anyway, signals aren't really needed any >>>>> more and just produce a lot of obtuse difficult to maintain code. So, I don't >>>>> see any way you can do this using signals but Lukasz may have a way of doing >>>>> this otherwise, However why are you using sync reads anyway? If this is a >>>>> problem you should be using async reads. >>>> >>>> No, we'll do async reads :) I know signals and thread-cancellation are a sure >>>> road into hell. >>>> >>>> Then, what happens when XrdCl::Close() is called in one thread? Will current >>>> reads fail? Or will Close() wait for dispatched transactions to complete? >>>> >>>> Matevz >>>> >>>>> Andy >>>>> >>>>> -----Original Message----- From: Matevz Tadel >>>>> Sent: Monday, March 17, 2014 2:19 PM >>>>> To: xrootd-dev >>>>> Subject: XrdCl interrupting sync read >>>>> >>>>> Hi, >>>>> >>>>> I tried to figure out how sigs are masked and handled in xrootd -- but they >>>>> don't seem to be used at all (or they are wrapped very well :) ) -- so >>>>> maybe the >>>>> whole thing below is a nono. >>>>> >>>>> Is it safe to signal a thread doing a synchronous XrdClFile::Read() to make it >>>>> come back with the equivalent of EINTR? What signal should I use for that? >>>>> >>>>> The thread is owned by me ... so I can install signal handlers if it's needed. >>>>> >>>>> Cheers, >>>>> Matevz >>>>> >>>>> ######################################################################## >>>>> Use REPLY-ALL to reply to list >>>>> >>>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>>> >>>> ######################################################################## >>>> Use REPLY-ALL to reply to list >>>> >>>> To unsubscribe from the XROOTD-DEV list, click the following link: >>>> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >>> >> >> ######################################################################## >> Use REPLY-ALL to reply to list >> >> To unsubscribe from the XROOTD-DEV list, click the following link: >> https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1 >> ######################################################################## Use REPLY-ALL to reply to list To unsubscribe from the XROOTD-DEV list, click the following link: https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1