Hi Brian, Vincenzo,
On Thu, Dec 22, 2011 at 03:32:54PM -0600, Brian Bockelman wrote:
> In CMS, Vincenzo has reported that using xrootd causes interesting memory spikes in our application. He wrote up a page here:
>
> https://twiki.cern.ch/twiki/bin/view/LCG/VIJemalloc
>
> Basically, the conclusion is that by switching malloc implementations, the memory spikes (which are large - 500MB of RAM!) go away. This likely indicates not memory leaks, but "abusive" memory allocation patterns or strange threading affects.
>
> Any idea where these might be coming from? Anything we can do to adjust this?
The thought was that this was due to some bad interaction between memory
arenas and the extra thread from the xrootd client.
As I suggested in the CMS HN thread about this, did someone follow up
on the variables that can be used to change that with the existing memory
allocator? (The relevant bit from that CMS HN thread is included below.)
Pete
On Wed, Dec 14, 2011 at 04:12:30PM +0100, Peter Elmer wrote:
> Hi Vincenzo,
>
> On Wed, Dec 14, 2011 at 04:08:17PM +0100, Vincenzo Innocente wrote:
> > On 14 Dec, 2011, at 4:05 PM, Vincenzo Innocente wrote:
> > > On 14 Dec, 2011, at 4:04 PM, Peter Elmer wrote:
> > >> I ran across this posting:
> > >>
> > >> http://www.quora.com/Why-do-some-applications-use-significantly-more-virtual-memory-on-RHEL-6-compared-to-RHEL-5
> > > on SLC5. I can also run on SLC6 if needed.
> > btw per thread arena is what jemalloc uses ans it seems to help to keep xrootd memory under control.
>
> Hmm, I wonder if we can achieve the same effect on SLC5 via the
> MALLOC_ARENA_* variables described here:
>
> http://www.centos.org/docs/5/html/5.4/technical-notes/glibc.html
>
> (see very last point in the 1.71.1 list, here for CentOS 5.4, but
> presumably it applies also RHEL5.4/SL5.4).
>
> Pete
-------------------------------------------------------------------------
Peter Elmer E-mail: [log in to unmask] Phone: +41 (22) 767-4644
Address: CERN Division PPE, Bat. 32 2C-14, CH-1211 Geneva 23, Switzerland
-------------------------------------------------------------------------
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
