[NCLUG] Re: malloc performance

Douglas Baskins dougbaskins at frii.com
Mon Aug 12 00:22:16 MDT 2002


> > [Doug Baskins] 
> > I am about to write a universal time/test program for malloc 

> [Doug Lea] 
> I'm pretty sure there's no universal test. There are better and worse
> choices of test suites though. Different malloc-intensive programs
> tend to have very different behaviors, and my released versions try to
> make sure that they all run well. (For some graphs of different kinds
> of programs, see http://gee.cs.oswego.edu/dl/malloc-plots/index.html)
> When I find cases that don't, I try to improve algorithms.
> 
> People regularly write papers comparing mallocs.  The most recent I
> know of is Emery Berger's "REAP" paper, that you can find at:
>   http://www.cs.utexas.edu/users/emery/
> (Note: These results will lead to inclusion of some region support
> in a forthcoming version of my malloc.) See also his references.
> 
> Still, I'd very much like to get a copy of your new test program!

Ok, I am almost done.  BTW, thanks for fixing the bug.  I will try it
tomorrow.

> And if anyone knows of some malloc-bound python programs out there
> that are amenable for use in a test suite, I'd be very interested in
> getting them.
> 
> -Doug Lea
 
Actually the way the "universal" time/test program works is:

1) Instrument your "problem" program to output a message for
   all malloc()s and free()s with a string that looks like:

   a 0x40123456 123    where 'a' means malloc(123) 
   f 0x40123456        where 'f' means free(0x40123456); 

2) Run the program and save output.

3) Run my test program with the file as input.
   The test program will process the input and "do some stuff"
   then do all the malloc()s and free()s of the same size and order
   as the origional program.  It will measure the performance free
   of most other overhead.

The test program also checks for memory overlaps, double free()s etc.
So far I have tried it with up to 35 million calls.  The trace file 
was over 1Gb in size, so I can't put it on my web site (my connection
is at 26Kbs.  But I can give you instructions on how to generate 
the trace.  The output is also suitable for ploting.

Doug Baskins <doug at sourcejudy.com>



More information about the NCLUG mailing list