libslack(mem) - memory module
#include <slack/mem.h>
typedef struct Pool Pool;
#define null NULL #define nul '\0'
#define mem_new(type) #define mem_create(size, type) #define mem_resize(mem, size) void *mem_resize_fn(void **mem, size_t size); #define mem_release(mem) void *mem_destroy(void **mem); void *mem_create_secure(size_t size); void mem_release_secure(void *mem); void *mem_destroy_secure(void **mem); char *mem_strdup(const char *str); #define mem_create2d(type, x, y) #define mem_create3d(type, x, y, z) #define mem_create4d(type, x, y, z, a) void *mem_create_space(size_t size, ...); size_t mem_space_start(size_t size, ...); #define mem_release2d(space) #define mem_release3d(space) #define mem_release4d(space) #define mem_release_space(space) #define mem_destroy2d(space) #define mem_destroy3d(space) #define mem_destroy4d(space) #define mem_destroy_space(space) Pool *pool_create(size_t size); Pool *pool_create_locked(Locker *locker, size_t size); void pool_release(Pool *pool); void *pool_destroy(Pool **pool); Pool *pool_create_secure(size_t size); Pool *pool_create_secure_locked(Locker *locker, size_t size); void pool_release_secure(Pool *pool); #define pool_destroy_secure(pool) void *pool_destroy_secure_fn(Pool **pool); void pool_clear_secure(Pool *pool); #define pool_new(pool, type) #define pool_newsz(pool, size, type) void *pool_alloc(Pool *pool, size_t bytes); void pool_clear(Pool *pool);
This module is mostly just an interface to malloc(3), realloc(3) and
free(3) that tries to ensure that pointers that don't point to anything get set to NULL
. It also provides dynamically allocated multi-dimensional arrays, memory
pools and secure memory for the more adventurous.
#define null NULL
A less angular version of NULL
.
#define nul '\0'
A name for the nul
character.
#define mem_new(type)
Allocates enough memory (with malloc(3)) to store an object of type
type
. On success, returns the address of the allocated memory. On error,
returns NULL
.
#define mem_create(size, type)
Allocates enough memory (with malloc(3)) to store size
objects of type
type
. On success, returns the address of the allocated memory. On error,
returns NULL
.
#define mem_resize(mem, num)
Alters the amount of memory pointed to by *mem
. If *mem
is NULL
,
malloc(3) is used to allocate new memory. If size is zero, free(3) is called to deallocate the memory and *mem
is set to NULL
. Otherwise,
realloc(3) is called. If realloc(3) needs to allocate new memory to satisfy a request, *mem
is set to the new address. On success, returns
*mem (though it may be NULL
if size
is zero). On error, NULL
is returned and *mem is not altered.
void *mem_resize_fn(void **mem, size_t size)
A single interface for altering the size of allocated memory. mem
points to the pointer to be affected. size
is the size in bytes of the memory that this pointer is to point to. If the
pointer is NULL
, malloc(3) is used to obtain memory. If size
is zero, free(3) is used to release the memory. In all other cases, realloc(3) is used to alter the size of the memory. In all cases, the pointer pointed
to by mem
is assigned to the memory's location (or NULL
when size
is zero). This function is exposed as an implementation side effect. Don't
call it directly. Call
mem_resize() instead. On error (i.e. malloc(3) or realloc(3) fail or mem
is NULL
), returns NULL
without setting *mem
to anything.
#define mem_release(mem)
Releases (deallocates) mem
. Same as free(3). Only to be used in destructor functions. In other cases, use mem_destroy() which also sets
mem
to NULL
.
void *mem_destroy(void **mem)
Calls free(3) on the pointer pointed to by *mem
. Then assigns NULL
to this pointer. Returns NULL
.
void *mem_create_secure(size_t size)
Allocates size
bytes of memory (with malloc(3)) and then locks it into RAM with mlock(2) so that it can't be paged to disk where some nefarious local user with root
access might read its contents. The memory returned must only be
deallocated using mem_release_secure() or mem_destroy_secure()
which will clear the memory and unlock it before deallocating it. On
success, returns the address of the secure allocated memory. On error,
returns NULL
with errno
set appropriately.
Note that entire pages are locked by mlock(2) so don't create many, small pieces of secure memory or many entire pages will be locked. Use a secure memory pool instead. Also note that secure memory requires root privileges.
On some systems (e.g. Solaris), memory locks must start on page boundaries.
So we need to malloc()
enough memory to extend from whatever address
malloc()
may return to the next page boundary (worst case: pagesize -
sizeof(int)
) and then the actual number of bytes requested. We need an additional 8
bytes to store the address returned by malloc()
(so we can
free()
it later) and the size passed to mlock()
so we can pass it to
munlock()
later. Unfortunately, we need to store the address and size after the page
boundary and not before it because malloc()
may return a page boundary or an address less than 8 bytes to the left of a
page boundary.
It will look like:
for free() +-------+ +- size+8 for munlock() v | v +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |* * * *|# # # #| | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ^ ^ ^ . . . size bytes . . . . . . | +- next page | +- malloc() +- address returned
If your system doesn't require page boundaries (e.g. Linux), the address
returned by malloc()
is locked and returned and only the size is stored.
void mem_release_secure(void *mem)
Sets the contents of mem
to nul
bytes, then unlocks and releases (deallocates) mem
. Only to be used on memory returned by
mem_create_secure(). Only to be used in destructor functions. In other cases, use mem_destroy() which also sets mem
to NULL
.
void *mem_destroy_secure(void **mem)
Sets the contents of *mem
to nul
bytes, then unlocks and destroys (deallocates and sets to NULL
) it. Only to be used on memory returned by
mem_create_secure(). Returns NULL
.
char *mem_strdup(const char *str)
Returns a dynamically allocated copy of str
. On error, returns NULL
. The caller must deallocate the memory returned.
#define mem_create2d(i, j, type)
Shorthand for allocating a 2-dimensional array. See mem_create_space().
#define mem_create3d(i, j, k, type)
Shorthand for allocating a 3-dimensional array. See mem_create_space().
#define mem_create4d(i, j, k, l, type)
Shorthand for allocating a 4-dimensional array. See mem_create_space().
void *mem_create_space(size_t size, ...)
Allocates a multi-dimensional array of elements of size size
and sets the memory to zero. The remaining arguments specify the sizes of
each dimension. The last argument must be zero. There is an arbitrary limit
of 32 dimensions. The memory returned is set to zero. The memory returned
needs to be cast or assigned into the appropriate pointer type. You can
then set and access elements exactly like a real multi-dimensional C array.
Finally, it must be deallocated with mem_destroy_space() or mem_release_space() or
mem_destroy() or mem_release() or free(3).
Note: You must not use memset(3) on all of the returned memory because the start of this memory contains pointers into the remainder. The exact amount of this overhead depends on the number and size of dimensions. The memory is allocated with calloc(3) to reduce the need to memset(3) the elements but if you need to know where the elements begin, use mem_space_start().
The memory returned looks like (e.g.):
char ***a = mem_create3d(2, 2, 3, char);
+-------------------------+ +-------|-------------------+ | a +-------|-------|-------------+ | | | +-------|-------|-------|-------+ | | | v | | | | V V V V +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | a[0] | a[1] |a[0][0]|a[0][1]|a[1][0]|a[1][1]| | | | | | | | | | | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ^ ^ a a a a a a a a a a a a +-------|-------+ | 0 0 0 0 0 0 1 1 1 1 1 1 +-----------------------+ 0 0 0 1 1 1 0 0 0 1 1 1 0 1 2 0 1 2 0 1 2 0 1 2
size_t mem_space_start(size_t size, ...)
Calculates the amount of overhead required for a multi-dimensional array created by a call to mem_create_space() with the same arguments. If you need reset all elements in such an array to zero:
int ****space = mem_create_space(sizeof(int), 2, 3, 4, 5, 0); size_t start = mem_space_start(sizeof(int), 2, 3, 4, 5, 0); memset((char *)space + start, '\0', sizeof(int) * 2 * 3 * 4 * 5);
#define mem_release2d(space)
Alias for releasing (deallocating) a 2-dimensional array. See mem_release_space().
#define mem_release3d(space)
Alias for releasing (deallocating) a 3-dimensional array. See mem_release_space().
#define mem_release4d(space)
Alias for releasing (deallocating) a 4-dimensional array. See mem_release_space().
#define mem_release_space(space)
Releases (deallocates) a multi-dimensional array, space
, allocated with
mem_create_space. Same as free(3). Only to be used in destructor functions. In other cases, use mem_destroy_space() which also sets
space
to NULL
.
#define mem_destroy2d(space)
Alias for destroying (deallocating and setting to NULL
) a 2-dimensional array. See mem_destroy_space().
#define mem_destroy3d(space)
Alias for destroying (deallocating and setting to NULL
) a 3-dimensional array. See mem_destroy_space().
#define mem_destroy4d(space)
Alias for destroying (deallocating and setting to NULL
) a 4-dimensional array. See mem_destroy_space().
#define mem_destroy_space(mem)
Destroys (deallocates and sets to NULL
) the multi-dimensional array pointed to by space
.
Pool *pool_create(size_t size)
Creates a memory pool of size size
from which many smaller chunks of memory may be subsequently allocated
(with pool_alloc()) without resorting to the use of malloc(3)
. Useful when you have many small objects to allocate but malloc(3) is slowing your program down too much. On success, returns the pool. On
error, returns NULL
.
The size of a pool can't be changed after it is created and the individual chunks of memory allocated from within a pool can't be separately deallocated. The entire pool can be emptied with pool_clear() and the pool can be deallocated with pool_release() or pool_destroy().
Pool *pool_create_locked(Locker *locker, size_t size)
Just like pool_alloc() except that multiple threads accessing this pool will be synchronised by locker
. On success, returns the pool. On error, returns NULL
.
void pool_release(Pool *pool)
Releases (deallocates) pool
. Only to be used in destructor functions. In other cases, use pool_destroy() which also sets pool
to NULL
.
void *pool_destroy(Pool **pool)
Destroys (deallocates and sets to NULL
) *pool
. Returns NULL
.
Note: pools shared by multiple threads must not be destroyed until after the
threads have finished with it.
Pool *pool_create_secure(size_t size)
Creates a memory pool of size size
just like pool_create() except that the memory pool itself is locked into RAM with mlock(2) so that it can't be paged to disk where some nefarious local user might
read its contents. The pool returned must only be deallocated using pool_release_secure() or
pool_destroy_secure() which will clear the memory pool and unlock it before deallocating it. In
all other ways, the pool returned is exactly like a pool returned by pool_create(). On success, returns the pool. On error, returns NULL
with errno
set appropriately. Note that secure memory requires root privileges.
Pool *pool_create_secure_locked(Locker *locker, size_t size)
Just like pool_create_secure() except that multiple threads accessing this pool will be synchronised by locker.
void pool_release_secure(Pool *pool)
Sets the contents of the memory pool to nul
bytes, then unlocks and releases (deallocates) pool
. Only to be used on pools returned by
pool_create_secure(). Only to be used in destructor functions. In other cases, use pool_destroy_secure() which also sets pool
to NULL
.
void *pool_destroy_secure(Pool **pool)
Sets the contents of the memory pool to nul
bytes, then unlocks and destroyd (deallocates and sets to NULL
) C<*pool). Returns NULL
.
Note: secure pools shared by multiple threads must not be destroyed until after
the threads have finished with it.
void pool_clear_secure(Pool *pool)
Fills the secure pool
with nul
bytes and deallocates all of the chunks of secure memory previously
allocated from pool
so that it can be reused. Does not use free(3).
#define pool_new(pool, type)
Allocates enough memory from pool
to store an object of type type
. On success, returns the address of the allocated memory. On error,
returns
NULL
with errno
set appropriately.
#define pool_newsz(pool, size, type)
Allocates enough memory from pool
to store size
objects of type
type
. On success, returns the address of the allocated memory. On error,
returns NULL
with errno
set appropriately.
void *pool_alloc(Pool *pool, size_t size)
Allocates a chunk of memory of size
bytes from pool
. Does not use
malloc(3)
. The pointer returned must not be passed to free(3) or
realloc(3)
. Only the entire pool can be deallocated with
pool_release() or pool_destroy(). All of the chunks can be deallocated in one go with pool_clear() without deallocating the pool itself.
On success, returns the pointer to the allocated pool memory. On error,
returns NULL
with errno
set appropriately (i.e. EINVAL
if pool
is NULL
, ENOSPC if pool
does not have enough unused memory to allocate size
bytes).
It is the caller's responsibility to ensure the correct alignment if necessary by allocating the right numbers of bytes. The easiest way to do ensure is to use separate pools for each specific data type that requires specific alignment.
void pool_clear(Pool *pool)
Deallocates all of the chunks of memory previously allocated from pool
so that it can be reused. Does not use free(3).
MT-Safe (mem)
MT-Disciplined (pool) man thread(3) for details.
1D array of longs:
long *mem = mem_create(100, long); mem_resize(&mem, 200); mem_destroy(&mem);
3D array of ints:
int ***space = mem_create3d(10, 20, 30, int); int i, j, k;
for (i = 0; i < 10; ++i) for (j = 0; j < 20; ++j) for (k = 0; k < 30; ++k) space[i][j][k] = i + j + j;
mem_destroy3d(&space);
A pool of a million integers:
void pool() { Pool *pool; int i, *p; pool = pool_create(1024 * 1024 * sizeof(int)); if (!pool) return;
for (i = 0; i < 1024 * 1024; ++i) { p = pool_new(pool, int); *p = i; }
pool_destroy(&pool); }
Secure memory:
char *secure_passwd = mem_create_secure(32); if (!secure_passwd) exit(1); get_passwd(secure_passwd, 32); use_passwd(secure_passwd); mem_destroy_secure(&secure_passwd);
Secure memory pool:
Pool *secure_pool; char *secure_passwd; secure_pool = pool_create_secure(1024 * 1024); if (!secure_pool) exit(1); secure_passwd = pool_alloc(secure_pool, 32); get_passwd(secure_passwd, 32); use_passwd(secure_passwd); pool_destroy_secure(&secure_pool);
libslack(3), malloc(3), realloc(3), calloc(3), free(3), mlock(2), thread(3)
20010215 raf <raf@raf.org>