Allocating memory on pages, mapped into process memory on demand

From: Wolfgang Draxinger (wdraxinger_at_darkstargames.de)
Date: 07/31/04 

Date: Sat, 31 Jul 2004 14:21:11 +0200

<veröffentlicht & per Mail versendet>

Currently I'm developing a interactive application (3D game
engine) that deals with many lage objects, with very different
sizes. It consumes about 300MB to 500MB of memory at runtime.
My Problem is, that the app suffers from memory fragmentation,
making it neccesary, to "clean up" the address space now and
then, usually when a malloc fails, but there's still enough free
space around. However this causes a very distracting lag.

>From Windows I know the following scheme, I used in the Win32
port:

Most of the large memory blocks contain data, that is needed only
at very specific points in the engine's processing pipeline. At
any other time these objects do not to be directly accessible.

What I do is, to tell the OS, that I want n bytes of memory and
get a handle to this. When needing access to the memory I tell
it the OS to have it mapped, maybe at a chooseable address, in
the processes memory space so that I can access it.

Within the Win32 API it is possible to do this, using the
functions
| VirtualAlloc[Ex]
| AllocateUserPhysicalPages
| FreeUserPhysicalPages
| MapUserPhysicakPages[Scatter]
which are precicely designed for this purpose.

However Linux is my preferred OS, but until now I had never the
situation, that dynamic memory as I get it from malloc has this
fragmentation rate, so I never have extensively RTFM about user
space memory management.

Would be really nice, to have me redirected to the right manuals.
The names of the functions to call would be enough.

Thank you in advance.

Wolfgang