Subversion Repositories Kolibri OS

* Vital statistics:

  Supported pointer/size_t representation:       4 or 8 bytes

       size_t MUST be an unsigned type of the same width as

       pointers. (If you are using an ancient system that declares

       size_t as a signed type, or need it to be a different width

       than pointers, you can use a previous release of this malloc

       (e.g. 2.7.2) supporting these.)

  Alignment:                                     8 bytes (minimum)

       This suffices for nearly all current machines and C compilers.

       However, you can define MALLOC_ALIGNMENT to be wider than this

       if necessary (up to 128bytes), at the expense of using more space.

  Minimum overhead per allocated chunk:   4 or  8 bytes (if 4byte sizes)

                                          8 or 16 bytes (if 8byte sizes)

       Each malloced chunk has a hidden word of overhead holding size

       and status information, and additional cross-check word

       if FOOTERS is defined.

  Minimum allocated size: 4-byte ptrs:  16 bytes    (including overhead)

                          8-byte ptrs:  32 bytes    (including overhead)

       Even a request for zero bytes (i.e., malloc(0)) returns a

       pointer to something of the minimum allocatable size.

       The maximum overhead wastage (i.e., number of extra bytes

       allocated than were requested in malloc) is less than or equal

       to the minimum size, except for requests >= mmap_threshold that

       are serviced via mmap(), where the worst case wastage is about

       32 bytes plus the remainder from a system page (the minimal

       mmap unit); typically 4096 or 8192 bytes.

  Security: static-safe; optionally more or less

       The "security" of malloc refers to the ability of malicious

       code to accentuate the effects of errors (for example, freeing

       space that is not currently malloc'ed or overwriting past the

       ends of chunks) in code that calls malloc.  This malloc

       guarantees not to modify any memory locations below the base of

       heap, i.e., static variables, even in the presence of usage

       errors.  The routines additionally detect most improper frees

       and reallocs.  All this holds as long as the static bookkeeping

       for malloc itself is not corrupted by some other means.  This

       is only one aspect of security -- these checks do not, and

       cannot, detect all possible programming errors.

       If FOOTERS is defined nonzero, then each allocated chunk

       carries an additional check word to verify that it was malloced

       from its space.  These check words are the same within each

       execution of a program using malloc, but differ across

       executions, so externally crafted fake chunks cannot be

       freed. This improves security by rejecting frees/reallocs that

       could corrupt heap memory, in addition to the checks preventing

       writes to statics that are always on.  This may further improve

       security at the expense of time and space overhead.  (Note that

       FOOTERS may also be worth using with MSPACES.)

       By default detected errors cause the program to abort (calling

       "abort()"). You can override this to instead proceed past

       errors by defining PROCEED_ON_ERROR.  In this case, a bad free

       has no effect, and a malloc that encounters a bad address

       caused by user overwrites will ignore the bad address by

       dropping pointers and indices to all known memory. This may

       be appropriate for programs that should continue if at all

       possible in the face of programming errors, although they may

       run out of memory because dropped memory is never reclaimed.

       If you don't like either of these options, you can define

       CORRUPTION_ERROR_ACTION and USAGE_ERROR_ACTION to do anything

       else. And if if you are sure that your program using malloc has

       no errors or vulnerabilities, you can define INSECURE to 1,

       which might (or might not) provide a small performance improvement.

       It is also possible to limit the maximum total allocatable

       space, using malloc_set_footprint_limit. This is not

       designed as a security feature in itself (calls to set limits

       are not screened or privileged), but may be useful as one

       aspect of a secure implementation.

  Thread-safety: NOT thread-safe unless USE_LOCKS defined non-zero

       When USE_LOCKS is defined, each public call to malloc, free,

       etc is surrounded with a lock. By default, this uses a plain

       pthread mutex, win32 critical section, or a spin-lock if if

       available for the platform and not disabled by setting

       USE_SPIN_LOCKS=0.  However, if USE_RECURSIVE_LOCKS is defined,

       recursive versions are used instead (which are not required for

       base functionality but may be needed in layered extensions).

       Using a global lock is not especially fast, and can be a major

       bottleneck.  It is designed only to provide minimal protection

       in concurrent environments, and to provide a basis for

       extensions.  If you are using malloc in a concurrent program,

       consider instead using nedmalloc

       (http://www.nedprod.com/programs/portable/nedmalloc/) or

       ptmalloc (See http://www.malloc.de), which are derived from

       versions of this malloc.

  System requirements: Any combination of MORECORE and/or MMAP/MUNMAP

       This malloc can use unix sbrk or any emulation (invoked using

       the CALL_MORECORE macro) and/or mmap/munmap or any emulation

       (invoked using CALL_MMAP/CALL_MUNMAP) to get and release system

       memory.  On most unix systems, it tends to work best if both

       MORECORE and MMAP are enabled.  On Win32, it uses emulations

       based on VirtualAlloc. It also uses common C library functions

       like memset.

  Compliance: I believe it is compliant with the Single Unix Specification

       (See http://www.unix.org). Also SVID/XPG, ANSI C, and probably

       others as well.

* Overview of algorithms

  This is not the fastest, most space-conserving, most portable, or

  most tunable malloc ever written. However it is among the fastest

  while also being among the most space-conserving, portable and

  tunable.  Consistent balance across these factors results in a good

  general-purpose allocator for malloc-intensive programs.

  In most ways, this malloc is a best-fit allocator. Generally, it

  chooses the best-fitting existing chunk for a request, with ties

  broken in approximately least-recently-used order. (This strategy

  normally maintains low fragmentation.) However, for requests less

  than 256bytes, it deviates from best-fit when there is not an

  exactly fitting available chunk by preferring to use space adjacent

  to that used for the previous small request, as well as by breaking

  ties in approximately most-recently-used order. (These enhance

  locality of series of small allocations.)  And for very large requests

  (>= 256Kb by default), it relies on system memory mapping

  facilities, if supported.  (This helps avoid carrying around and

  possibly fragmenting memory used only for large chunks.)

  All operations (except malloc_stats and mallinfo) have execution

  times that are bounded by a constant factor of the number of bits in

  a size_t, not counting any clearing in calloc or copying in realloc,

  or actions surrounding MORECORE and MMAP that have times

  proportional to the number of non-contiguous regions returned by

  system allocation routines, which is often just 1. In real-time

  applications, you can optionally suppress segment traversals using

  NO_SEGMENT_TRAVERSAL, which assures bounded execution even when

  system allocators return non-contiguous spaces, at the typical

  expense of carrying around more memory and increased fragmentation.

  The implementation is not very modular and seriously overuses

  macros. Perhaps someday all C compilers will do as good a job

  inlining modular code as can now be done by brute-force expansion,

  but now, enough of them seem not to.

  Some compilers issue a lot of warnings about code that is

  dead/unreachable only on some platforms, and also about intentional

  uses of negation on unsigned types. All known cases of each can be

  ignored.

  For a longer but out of date high-level description, see

     http://gee.cs.oswego.edu/dl/html/malloc.html

* MSPACES

  If MSPACES is defined, then in addition to malloc, free, etc.,

  this file also defines mspace_malloc, mspace_free, etc. These

  are versions of malloc routines that take an "mspace" argument

  obtained using create_mspace, to control all internal bookkeeping.

  If ONLY_MSPACES is defined, only these versions are compiled.

  So if you would like to use this allocator for only some allocations,

  and your system malloc for others, you can compile with

  ONLY_MSPACES and then do something like...

    static mspace mymspace = create_mspace(0,0); // for example

    #define mymalloc(bytes)  mspace_malloc(mymspace, bytes)

  (Note: If you only need one instance of an mspace, you can instead

  use "USE_DL_PREFIX" to relabel the global malloc.)

  You can similarly create thread-local allocators by storing

  mspaces as thread-locals. For example:

    static __thread mspace tlms = 0;

    void*  tlmalloc(size_t bytes) {

      if (tlms == 0) tlms = create_mspace(0, 0);

      return mspace_malloc(tlms, bytes);

    }

    void  tlfree(void* mem) { mspace_free(tlms, mem); }

  Unless FOOTERS is defined, each mspace is completely independent.

  You cannot allocate from one and free to another (although

  conformance is only weakly checked, so usage errors are not always

  caught). If FOOTERS is defined, then each chunk carries around a tag

  indicating its originating mspace, and frees are directed to their

  originating spaces. Normally, this requires use of locks.

 -------------------------  Compile-time options ---------------------------

Be careful in setting #define values for numerical constants of type

size_t. On some systems, literal values are not automatically extended

to size_t precision unless they are explicitly casted. You can also

use the symbolic values MAX_SIZE_T, SIZE_T_ONE, etc below.

WIN32                    default: defined if _WIN32 defined

  Defining WIN32 sets up defaults for MS environment and compilers.

  Otherwise defaults are for unix. Beware that there seem to be some

  cases where this malloc might not be a pure drop-in replacement for

  Win32 malloc: Random-looking failures from Win32 GDI API's (eg;

  SetDIBits()) may be due to bugs in some video driver implementations

  when pixel buffers are malloc()ed, and the region spans more than

  one VirtualAlloc()ed region. Because dlmalloc uses a small (64Kb)

  default granularity, pixel buffers may straddle virtual allocation

  regions more often than when using the Microsoft allocator.  You can

  avoid this by using VirtualAlloc() and VirtualFree() for all pixel

  buffers rather than using malloc().  If this is not possible,

  recompile this malloc with a larger DEFAULT_GRANULARITY. Note:

  in cases where MSC and gcc (cygwin) are known to differ on WIN32,

  conditions use _MSC_VER to distinguish them.

DLMALLOC_EXPORT       default: extern

  Defines how public APIs are declared. If you want to export via a

  Windows DLL, you might define this as

    #define DLMALLOC_EXPORT extern  __declspec(dllexport)

  If you want a POSIX ELF shared object, you might use

    #define DLMALLOC_EXPORT extern __attribute__((visibility("default")))

MALLOC_ALIGNMENT         default: (size_t)(2 * sizeof(void *))

  Controls the minimum alignment for malloc'ed chunks.  It must be a

  power of two and at least 8, even on machines for which smaller

  alignments would suffice. It may be defined as larger than this

  though. Note however that code and data structures are optimized for

  the case of 8-byte alignment.

MSPACES                  default: 0 (false)

  If true, compile in support for independent allocation spaces.

  This is only supported if HAVE_MMAP is true.

ONLY_MSPACES             default: 0 (false)

  If true, only compile in mspace versions, not regular versions.

USE_LOCKS                default: 0 (false)

  Causes each call to each public routine to be surrounded with

  pthread or WIN32 mutex lock/unlock. (If set true, this can be

  overridden on a per-mspace basis for mspace versions.) If set to a

  non-zero value other than 1, locks are used, but their

  implementation is left out, so lock functions must be supplied manually,

  as described below.

USE_SPIN_LOCKS           default: 1 iff USE_LOCKS and spin locks available

  If true, uses custom spin locks for locking. This is currently

  supported only gcc >= 4.1, older gccs on x86 platforms, and recent

  MS compilers.  Otherwise, posix locks or win32 critical sections are

  used.

USE_RECURSIVE_LOCKS      default: not defined

  If defined nonzero, uses recursive (aka reentrant) locks, otherwise

  uses plain mutexes. This is not required for malloc proper, but may

  be needed for layered allocators such as nedmalloc.

LOCK_AT_FORK            default: not defined

  If defined nonzero, performs pthread_atfork upon initialization

  to initialize child lock while holding parent lock. The implementation

  assumes that pthread locks (not custom locks) are being used. In other

  cases, you may need to customize the implementation.

FOOTERS                  default: 0

  If true, provide extra checking and dispatching by placing

  information in the footers of allocated chunks. This adds

  space and time overhead.

INSECURE                 default: 0

  If true, omit checks for usage errors and heap space overwrites.

USE_DL_PREFIX            default: NOT defined

  Causes compiler to prefix all public routines with the string 'dl'.

  This can be useful when you only want to use this malloc in one part

  of a program, using your regular system malloc elsewhere.

MALLOC_INSPECT_ALL       default: NOT defined

  If defined, compiles malloc_inspect_all and mspace_inspect_all, that

  perform traversal of all heap space.  Unless access to these

  functions is otherwise restricted, you probably do not want to

  include them in secure implementations.

ABORT                    default: defined as abort()

  Defines how to abort on failed checks.  On most systems, a failed

  check cannot die with an "assert" or even print an informative

  message, because the underlying print routines in turn call malloc,

  which will fail again.  Generally, the best policy is to simply call

  abort(). It's not very useful to do more than this because many

  errors due to overwriting will show up as address faults (null, odd

  addresses etc) rather than malloc-triggered checks, so will also

  abort.  Also, most compilers know that abort() does not return, so

  can better optimize code conditionally calling it.

PROCEED_ON_ERROR           default: defined as 0 (false)

  Controls whether detected bad addresses cause them to bypassed

  rather than aborting. If set, detected bad arguments to free and

  realloc are ignored. And all bookkeeping information is zeroed out

  upon a detected overwrite of freed heap space, thus losing the

  ability to ever return it from malloc again, but enabling the

  application to proceed. If PROCEED_ON_ERROR is defined, the

  static variable malloc_corruption_error_count is compiled in

  and can be examined to see if errors have occurred. This option

  generates slower code than the default abort policy.

DEBUG                    default: NOT defined

  The DEBUG setting is mainly intended for people trying to modify

  this code or diagnose problems when porting to new platforms.

  However, it may also be able to better isolate user errors than just

  using runtime checks.  The assertions in the check routines spell

  out in more detail the assumptions and invariants underlying the

  algorithms.  The checking is fairly extensive, and will slow down

  execution noticeably. Calling malloc_stats or mallinfo with DEBUG

  set will attempt to check every non-mmapped allocated and free chunk

  in the course of computing the summaries.

ABORT_ON_ASSERT_FAILURE   default: defined as 1 (true)

  Debugging assertion failures can be nearly impossible if your

  version of the assert macro causes malloc to be called, which will

  lead to a cascade of further failures, blowing the runtime stack.

  ABORT_ON_ASSERT_FAILURE cause assertions failures to call abort(),

  which will usually make debugging easier.

MALLOC_FAILURE_ACTION     default: sets errno to ENOMEM, or no-op on win32

  The action to take before "return 0" when malloc fails to be able to

  return memory because there is none available.

HAVE_MORECORE             default: 1 (true) unless win32 or ONLY_MSPACES

  True if this system supports sbrk or an emulation of it.

MORECORE                  default: sbrk

  The name of the sbrk-style system routine to call to obtain more

  memory.  See below for guidance on writing custom MORECORE

  functions. The type of the argument to sbrk/MORECORE varies across

  systems.  It cannot be size_t, because it supports negative

  arguments, so it is normally the signed type of the same width as

  size_t (sometimes declared as "intptr_t").  It doesn't much matter

  though. Internally, we only call it with arguments less than half

  the max value of a size_t, which should work across all reasonable

  possibilities, although sometimes generating compiler warnings.

MORECORE_CONTIGUOUS       default: 1 (true) if HAVE_MORECORE

  If true, take advantage of fact that consecutive calls to MORECORE

  with positive arguments always return contiguous increasing

  addresses.  This is true of unix sbrk. It does not hurt too much to

  set it true anyway, since malloc copes with non-contiguities.

  Setting it false when definitely non-contiguous saves time

  and possibly wasted space it would take to discover this though.

MORECORE_CANNOT_TRIM      default: NOT defined

  True if MORECORE cannot release space back to the system when given

  negative arguments. This is generally necessary only if you are

  using a hand-crafted MORECORE function that cannot handle negative

  arguments.

NO_SEGMENT_TRAVERSAL       default: 0

  If non-zero, suppresses traversals of memory segments

  returned by either MORECORE or CALL_MMAP. This disables

  merging of segments that are contiguous, and selectively

  releasing them to the OS if unused, but bounds execution times.

HAVE_MMAP                 default: 1 (true)

  True if this system supports mmap or an emulation of it.  If so, and

  HAVE_MORECORE is not true, MMAP is used for all system

  allocation. If set and HAVE_MORECORE is true as well, MMAP is

  primarily used to directly allocate very large blocks. It is also

  used as a backup strategy in cases where MORECORE fails to provide

  space from system. Note: A single call to MUNMAP is assumed to be

  able to unmap memory that may have be allocated using multiple calls

  to MMAP, so long as they are adjacent.

HAVE_MREMAP               default: 1 on linux, else 0

  If true realloc() uses mremap() to re-allocate large blocks and

  extend or shrink allocation spaces.

MMAP_CLEARS               default: 1 except on WINCE.

  True if mmap clears memory so calloc doesn't need to. This is true

  for standard unix mmap using /dev/zero and on WIN32 except for WINCE.

USE_BUILTIN_FFS            default: 0 (i.e., not used)

  Causes malloc to use the builtin ffs() function to compute indices.

  Some compilers may recognize and intrinsify ffs to be faster than the

  supplied C version. Also, the case of x86 using gcc is special-cased

  to an asm instruction, so is already as fast as it can be, and so

  this setting has no effect. Similarly for Win32 under recent MS compilers.

  (On most x86s, the asm version is only slightly faster than the C version.)

malloc_getpagesize         default: derive from system includes, or 4096.

  The system page size. To the extent possible, this malloc manages

  memory from the system in page-size units.  This may be (and

  usually is) a function rather than a constant. This is ignored

  if WIN32, where page size is determined using getSystemInfo during

  initialization.

USE_DEV_RANDOM             default: 0 (i.e., not used)

  Causes malloc to use /dev/random to initialize secure magic seed for

  stamping footers. Otherwise, the current time is used.

NO_MALLINFO                default: 0

  If defined, don't compile "mallinfo". This can be a simple way

  of dealing with mismatches between system declarations and

  those in this file.

MALLINFO_FIELD_TYPE        default: size_t

  The type of the fields in the mallinfo struct. This was originally

  defined as "int" in SVID etc, but is more usefully defined as

  size_t. The value is used only if  HAVE_USR_INCLUDE_MALLOC_H is not set

NO_MALLOC_STATS            default: 0

  If defined, don't compile "malloc_stats". This avoids calls to

  fprintf and bringing in stdio dependencies you might not want.

REALLOC_ZERO_BYTES_FREES    default: not defined

  This should be set if a call to realloc with zero bytes should

  be the same as a call to free. Some people think it should. Otherwise,

  since this malloc returns a unique pointer for malloc(0), so does

  realloc(p, 0).

LACKS_UNISTD_H, LACKS_FCNTL_H, LACKS_SYS_PARAM_H, LACKS_SYS_MMAN_H

LACKS_STRINGS_H, LACKS_STRING_H, LACKS_SYS_TYPES_H,  LACKS_ERRNO_H

LACKS_STDLIB_H LACKS_SCHED_H LACKS_TIME_H  default: NOT defined unless on WIN32

  Define these if your system does not have these header files.

  You might need to manually insert some of the declarations they provide.

DEFAULT_GRANULARITY        default: page size if MORECORE_CONTIGUOUS,

                                system_info.dwAllocationGranularity in WIN32,

                                otherwise 64K.

      Also settable using mallopt(M_GRANULARITY, x)

  The unit for allocating and deallocating memory from the system.  On

  most systems with contiguous MORECORE, there is no reason to

  make this more than a page. However, systems with MMAP tend to

  either require or encourage larger granularities.  You can increase

  this value to prevent system allocation functions to be called so

  often, especially if they are slow.  The value must be at least one

  page and must be a power of two.  Setting to 0 causes initialization

  to either page size or win32 region size.  (Note: In previous

  versions of malloc, the equivalent of this option was called

  "TOP_PAD")

DEFAULT_TRIM_THRESHOLD    default: 2MB

      Also settable using mallopt(M_TRIM_THRESHOLD, x)

  The maximum amount of unused top-most memory to keep before

  releasing via malloc_trim in free().  Automatic trimming is mainly

  useful in long-lived programs using contiguous MORECORE.  Because

  trimming via sbrk can be slow on some systems, and can sometimes be

  wasteful (in cases where programs immediately afterward allocate

  more large chunks) the value should be high enough so that your

  overall system performance would improve by releasing this much

  memory.  As a rough guide, you might set to a value close to the

  average size of a process (program) running on your system.

  Releasing this much memory would allow such a process to run in

  memory.  Generally, it is worth tuning trim thresholds when a

  program undergoes phases where several large chunks are allocated

  and released in ways that can reuse each other's storage, perhaps

  mixed with phases where there are no such chunks at all. The trim

  value must be greater than page size to have any useful effect.  To

  disable trimming completely, you can set to MAX_SIZE_T. Note that the trick

  some people use of mallocing a huge space and then freeing it at

  program startup, in an attempt to reserve system memory, doesn't

  have the intended effect under automatic trimming, since that memory

  will immediately be returned to the system.

DEFAULT_MMAP_THRESHOLD       default: 256K

      Also settable using mallopt(M_MMAP_THRESHOLD, x)

  The request size threshold for using MMAP to directly service a

  request. Requests of at least this size that cannot be allocated

  using already-existing space will be serviced via mmap.  (If enough

  normal freed space already exists it is used instead.)  Using mmap

  segregates relatively large chunks of memory so that they can be

  individually obtained and released from the host system. A request

  serviced through mmap is never reused by any other request (at least

  not directly; the system may just so happen to remap successive

  requests to the same locations).  Segregating space in this way has

  the benefits that: Mmapped space can always be individually released

  back to the system, which helps keep the system level memory demands

  of a long-lived program low.  Also, mapped memory doesn't become

  `locked' between other chunks, as can happen with normally allocated

  chunks, which means that even trimming via malloc_trim would not

  release them.  However, it has the disadvantage that the space

  cannot be reclaimed, consolidated, and then used to service later

  requests, as happens with normal chunks.  The advantages of mmap

  nearly always outweigh disadvantages for "large" chunks, but the

  value of "large" may vary across systems.  The default is an

  empirically derived value that works well in most systems. You can

  disable mmap by setting to MAX_SIZE_T.

MAX_RELEASE_CHECK_RATE   default: 4095 unless not HAVE_MMAP

  The number of consolidated frees between checks to release

  unused segments when freeing. When using non-contiguous segments,

  especially with multiple mspaces, checking only for topmost space

  doesn't always suffice to trigger trimming. To compensate for this,

  free() will, with a period of MAX_RELEASE_CHECK_RATE (or the

  current number of segments, if greater) try to release unused

  segments to the OS when freeing chunks that result in

  consolidation. The best value for this parameter is a compromise

  between slowing down frees with relatively costly checks that

  rarely trigger versus holding on to unused memory. To effectively

  disable, set to MAX_SIZE_T. This may lead to a very slight speed

#ifndef DLMALLOC_VERSION

#define DLMALLOC_VERSION 20806

#endif /* DLMALLOC_VERSION */

/*

  malloc(size_t n)

  Returns a pointer to a newly allocated chunk of at least n bytes, or

  null if no space is available, in which case errno is set to ENOMEM

  on ANSI C systems.

  If n is zero, malloc returns a minimum-sized chunk. (The minimum

  size is 16 bytes on most 32bit systems, and 32 bytes on 64bit

  systems.)  Note that size_t is an unsigned type, so calls with

  arguments that would be negative if signed are interpreted as

  requests for huge amounts of space, which will often fail. The

  maximum supported value of n differs across systems, but is in all

  cases less than the maximum representable value of a size_t.

*/

/*

  free(void* p)

  Releases the chunk of memory pointed to by p, that had been previously

  allocated using malloc or a related routine such as realloc.

  It has no effect if p is null. If p was not malloced or already

  freed, free(p) will by default cause the current program to abort.

*/

/*

  calloc(size_t n_elements, size_t element_size);

  Returns a pointer to n_elements * element_size bytes, with all locations

  set to zero.

*/

/*

  realloc(void* p, size_t n)

  Returns a pointer to a chunk of size n that contains the same data

  as does chunk p up to the minimum of (n, p's size) bytes, or null

  if no space is available.

  The returned pointer may or may not be the same as p. The algorithm

  prefers extending p in most cases when possible, otherwise it

  employs the equivalent of a malloc-copy-free sequence.

  If p is null, realloc is equivalent to malloc.

  If space is not available, realloc returns null, errno is set (if on

  ANSI) and p is NOT freed.

  if n is for fewer bytes than already held by p, the newly unused

  space is lopped off and freed if possible.  realloc with a size

  argument of zero (re)allocates a minimum-sized chunk.

  The old unix realloc convention of allowing the last-free'd chunk

*/

/*

  memalign(size_t alignment, size_t n);

  Returns a pointer to a newly allocated chunk of n bytes, aligned

  in accord with the alignment argument.

  The alignment argument should be a power of two. If the argument is

  not a power of two, the nearest greater power is used.

  8-byte alignment is guaranteed by normal malloc calls, so don't

  bother calling memalign with an argument of 8 or less.

  Overreliance on memalign is a sure way to fragment space.

*/

#define DEBUG   1

#define HAVE_MMAP               1

/* -------------------------- MMAP preliminaries ------------------------- */

/*

   If HAVE_MORECORE or HAVE_MMAP are false, we just define calls and

   checks to fail so compiler optimizer can delete code rather than

   using so many "#if"s.

*/

#define CMFAIL               ((char*)(MFAIL)) /* defined for convenience */

#define should_trim(M,s)  (0)

/* --------------------------- Lock preliminaries ------------------------ */

/*

  When locks are defined, there is one global lock, plus

  one per-mspace lock.

  The global lock_ensures that mparams.magic and other unique

  mparams values are initialized only once. It also protects

  sequences of calls to MORECORE.  In many cases sys_alloc requires

  two calls, that should not be interleaved with calls by other

  threads.  This does not protect against direct calls to MORECORE

  by other threads not using this lock, so there is still code to

  cope the best we can on interference.

  Per-mspace locks surround calls to malloc, free, etc.

  By default, locks are simple non-reentrant mutexes.

  Because lock-protected regions generally have bounded times, it is

  OK to use the supplied simple spinlocks. Spinlocks are likely to

  improve performance for lightly contended applications, but worsen

  performance under heavy contention.

  If USE_LOCKS is > 1, the definitions of lock routines here are

  bypassed, in which case you will need to define the type MLOCK_T,

  and at least INITIAL_LOCK, DESTROY_LOCK, ACQUIRE_LOCK, RELEASE_LOCK

  and TRY_LOCK.  You must also declare a

    static MLOCK_T malloc_global_mutex = { initialization values };.

*/

static DEFINE_MUTEX(malloc_global_mutex);

#define ACQUIRE_MALLOC_GLOBAL_LOCK()  MutexLock(&malloc_global_mutex);

#define RELEASE_MALLOC_GLOBAL_LOCK()  MutexUnlock(&malloc_global_mutex);

/* -----------------------  Chunk representations ------------------------ */

/*

  (The following includes lightly edited explanations by Colin Plumb.)

  The malloc_chunk declaration below is misleading (but accurate and

  necessary).  It declares a "view" into memory allowing access to

  necessary fields at known offsets from a given base.

  originally described by Knuth.  (See the paper by Paul Wilson

  ftp://ftp.cs.utexas.edu/pub/garbage/allocsrv.ps for a survey of such

  techniques.)  Sizes of free chunks are stored both in the front of

  each chunk and at the end.  This makes consolidating fragmented

  chunks into bigger chunks fast.  The head fields also hold bits

  representing whether chunks are free or in use.

  Here are some pictures to make it clearer.  They are "exploded" to

  show that the state of a chunk can be thought of as extending from

  the high 31 bits of the head field of its header through the

  prev_foot and PINUSE_BIT bit of the following chunk header.

  A chunk that's in use looks like:

   chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

           | Size of previous chunk (if P = 0)                             |

           +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |P|

         | Size of this chunk                                         1| +-+

   mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         |                                                               |

         +-                                                             -+

         |                                                               |

         +-                                                             -+

         |                                                               :

         +-      size - sizeof(size_t) available payload bytes          -+

         :                                                               |

 chunk-> +-                                                             -+

         |                                                               |

         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1|

       | Size of next chunk (may or may not be in use)               | +-+

 mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    And if it's free, it looks like this:

   chunk-> +-                                                             -+

           | User payload (must be in use, or we would have merged!)       |

           +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |P|

         | Size of this chunk                                         0| +-+

   mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         | Next pointer                                                  |

         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         | Prev pointer                                                  |

         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         |                                                               :

         +-      size - sizeof(struct chunk) unused bytes               -+

         :                                                               |

 chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

         | Size of this chunk                                            |

         +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0|

       | Size of next chunk (must be in use, or we would have merged)| +-+

 mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       |                                                               :

       +- User payload                                                -+

       :                                                               |

       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                                                                     |0|

                                                                     +-+

  Note that since we always merge adjacent free chunks, the chunks

  adjacent to a free chunk must be in use.

  Given a pointer to a chunk (which can be derived trivially from the

  payload pointer) we can, in O(1) time, find out whether the adjacent

  chunks are free, and if so, unlink them from the lists that they

  are on and merge them with the current chunk.

  Chunks always begin on even word boundaries, so the mem portion

  (which is returned to the user) is also on an even word boundary, and

  thus at least double-word aligned.

  The P (PINUSE_BIT) bit, stored in the unused low-order bit of the

  chunk size (which is always a multiple of two words), is an in-use

  bit for the *previous* chunk.  If that bit is *clear*, then the

  word before the current chunk size contains the previous chunk

  size, and can be used to find the front of the previous chunk.

  The very first chunk allocated always has this bit set, preventing

  access to non-existent (or non-owned) memory. If pinuse is set for

  any given chunk, then you CANNOT determine the size of the

  previous chunk, and might even get a memory addressing fault when

  trying to do so.

  The C (CINUSE_BIT) bit, stored in the unused second-lowest bit of

  the chunk size redundantly records whether the current chunk is

  inuse (unless the chunk is mmapped). This redundancy enables usage

  checks within free and realloc, and reduces indirection when freeing

  and consolidating chunks.

  Each freshly allocated chunk must have both cinuse and pinuse set.

  That is, each allocated chunk borders either a previously allocated

  ensured by making all allocations from the `lowest' part of any

  found chunk.  Further, no free chunk physically borders another one,

  so each free chunk is known to be preceded and followed by either

  inuse chunks or the ends of memory.

  Note that the `foot' of the current chunk is actually represented

  as the prev_foot of the NEXT chunk. This makes it easier to

  deal with alignments etc but can be very confusing when trying

  to extend or adapt this code.

  The exceptions to all this are

     1. The special chunk `top' is the top-most available chunk (i.e.,

        the one bordering the end of available memory). It is treated

        specially.  Top is never included in any bin, is used only if

        no other chunk is available, and is released back to the

        system if it is very large (see M_TRIM_THRESHOLD).  In effect,

        the top chunk is treated as larger (and thus less well

        fitting) than any other available chunk.  The top chunk

        doesn't update its trailing size field since there is no next

        contiguous chunk that would have to index off it. However,

        space is still allocated for it (TOP_FOOT_SIZE) to enable

        separation or merging when space is extended.

     3. Chunks allocated via mmap, have both cinuse and pinuse bits

        cleared in their head fields.  Because they are allocated

        one-by-one, each must carry its own prev_foot field, which is

        also used to hold the offset this chunk has within its mmapped

        chunk is trailed by the first two fields of a fake next-chunk

*/

struct malloc_chunk {

  size_t               prev_foot;  /* Size of previous chunk (if free).  */

  size_t               head;       /* Size and inuse bits. */

  struct malloc_chunk* fd;         /* double links -- used only if free. */

  struct malloc_chunk* bk;

};

typedef struct malloc_chunk  mchunk;

typedef struct malloc_chunk* mchunkptr;

typedef unsigned int binmap_t;         /* Described below */

typedef unsigned int flag_t;           /* The type of various bit flag sets */

/* ------------------- Chunks sizes and alignments ----------------------- */

#define MCHUNK_SIZE         (sizeof(mchunk))

  (((req) < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(req))

#define pinuse(p)           ((p)->head & PINUSE_BIT)

#define chunksize(p)        ((p)->head & ~(FLAG_BITS))

#define next_pinuse(p)  ((next_chunk(p)->head) & PINUSE_BIT)

/* Get/set size at footer */

#define get_foot(p, s)  (((mchunkptr)((char*)(p) + (s)))->prev_foot)

 (is_mmapped(p)? MMAP_CHUNK_OVERHEAD : CHUNK_OVERHEAD)

/* Return true if malloced space is not necessarily cleared */

#if MMAP_CLEARS

#define calloc_must_clear(p) (!is_mmapped(p))

#else /* MMAP_CLEARS */

#define calloc_must_clear(p) (1)

#endif /* MMAP_CLEARS */

/* ---------------------- Overlaid data structures ----------------------- */

/*

  When chunks are not in use, they are treated as nodes of either

  lists or trees.

  "Small"  chunks are stored in circular doubly-linked lists, and look

  like this:

    chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             Size of previous chunk                            |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    `head:' |             Size of chunk, in bytes                         |P|

      mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             Forward pointer to next chunk in list             |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             Back pointer to previous chunk in list            |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             Unused space (may be 0 bytes long)                .

            .                                                               .

            .                                                               |

nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    `foot:' |             Size of chunk, in bytes                           |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Larger chunks are kept in a form of bitwise digital trees (aka

  tries) keyed on chunksizes.  Because malloc_tree_chunks are only for

  free chunks greater than 256 bytes, their size doesn't impose any

  constraints on user chunk sizes.  Each node looks like:

    chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             Size of previous chunk                            |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    `head:' |             Size of chunk, in bytes                         |P|

      mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             Forward pointer to next chunk of same size        |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             Back pointer to previous chunk of same size       |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             Pointer to left child (child[0])                  |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             Pointer to right child (child[1])                 |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             Pointer to parent                                 |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             bin index of this chunk                           |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

            |             Unused space                                      .

            .                                                               |

nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    `foot:' |             Size of chunk, in bytes                           |

            +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Each tree holding treenodes is a tree of unique chunk sizes.  Chunks

  of the same size are arranged in a circularly-linked list, with only

  the oldest chunk (the next to be used, in our FIFO ordering)

  actually in the tree.  (Tree members are distinguished by a non-null

  parent pointer.)  If a chunk with the same size an an existing node

  is inserted, it is linked off the existing node using pointers that

  work in the same way as fd/bk pointers of small chunks.

  Each tree contains a power of 2 sized range of chunk sizes (the

  smallest is 0x100 <= x < 0x180), which is is divided in half at each

  tree level, with the chunks in the smaller half of the range (0x100

  <= x < 0x140 for the top nose) in the left subtree and the larger

  half (0x140 <= x < 0x180) in the right subtree.  This is, of course,

  done by inspecting individual bits.

  Using these rules, each node's left subtree contains all smaller

  sizes than its right subtree.  However, the node at the root of each

  subtree has no particular ordering relationship to either.  (The

  dividing line between the subtree sizes is based on trie relation.)

  If we remove the last chunk of a given size from the interior of the

  tree, we need to replace it with a leaf node.  The tree ordering

  rules permit a node to be replaced by any leaf below it.

  The smallest chunk in a tree (a common operation in a best-fit

  allocator) can be found by walking a path to the leftmost leaf in

  the tree.  Unlike a usual binary tree, where we follow left child

  pointers until we reach a null, here we follow the right child

  pointer any time the left one is null, until we reach a leaf with

  both child pointers null. The smallest chunk in the tree will be

  somewhere along that path.

  The worst case number of steps to add, find, or remove a node is

  bounded by the number of bits differentiating chunks within

  bins. Under current bin calculations, this ranges from 6 up to 21

  (for 32 bit sizes) or up to 53 (for 64 bit sizes). The typical case

#define leftmost_child(t) ((t)->child[0] != 0? (t)->child[0] : (t)->child[1])

/* ----------------------------- Segments -------------------------------- */

/*

  Each malloc space may include non-contiguous segments, held in a

  list headed by an embedded malloc_segment record representing the

  top-most space. Segments also include flags holding properties of

  the space. Large chunks that are directly allocated by mmap are not

  included in this list. They are instead independently created and

  destroyed without otherwise keeping track of them.

  Segment management mainly comes into play for spaces allocated by

  MMAP.  Any call to MMAP might or might not return memory that is

  adjacent to an existing segment.  MORECORE normally contiguously

  extends the current space, so this space is almost always adjacent,

  which is simpler and faster to deal with. (This is why MORECORE is

  used preferentially to MMAP when both are available -- see

  sys_alloc.)  When allocating using MMAP, we don't use any of the

  hinting mechanisms (inconsistently) supported in various

  implementations of unix mmap, or distinguish reserving from

  committing memory. Instead, we just ask for space, and exploit

  contiguity when we get it.  It is probably possible to do

  better than this on some systems, but no general scheme seems

  to be significantly better.

  Management entails a simpler variant of the consolidation scheme

  used for chunks to reduce fragmentation -- new adjacent memory is

  normally prepended or appended to an existing segment. However,

  there are limitations compared to chunk consolidation that mostly

  reflect the fact that segment processing is relatively infrequent

  (occurring only when getting memory from system) and that we

  don't expect to have huge numbers of segments:

  * Segments are not indexed, so traversal requires linear scans.  (It

    would be possible to index these, but is not worth the extra

    overhead and complexity for most programs on most platforms.)

  * New segments are only appended to old ones when holding top-most

    memory; if they cannot be prepended to others, they are held in

    different segments.

  Except for the top-most segment of an mstate, each segment record

  is kept at the tail of its segment. Segments are added by pushing

  segment records onto the list headed by &mstate.seg for the

  containing mstate.

  Segment flags control allocation/merge/deallocation policies:

  * If EXTERN_BIT set, then we did not allocate this segment,

    and so should not try to deallocate or merge with others.

    (This currently holds only for the initial segment passed

    into create_mspace_with_base.)

  * If USE_MMAP_BIT set, the segment may be merged with

    other surrounding mmapped segments and trimmed/de-allocated

    using munmap.

  * If neither bit is set, then the segment was obtained using

    MORECORE so can be merged with surrounding MORECORE'd segments

    A cross-check field that should always hold same value as mparams.magic.

  Max allowed footprint

  size_t     footprint_limit; /* zero means no limit */

static struct malloc_state _gm_;

#define is_global(M)       ((M) == &_gm_)

#define is_initialized(M)  ((M)->top != 0)

/* -------------------------- system alloc setup ------------------------- */

/* Operations on mflags */