The following is taken straight out of the HeapAgent manual:
"HeapAgent causes a number of first-chance exceptions during its normal
operation. These are the result of address validation that HeapAgent performs
while validation parameters and querying the preexisting heap implementation
during heap patching. These first-chance exceptions are not errors in your application
or in HeapAgent. You can safely ignore them. You should therefore configure
your debugger to continue on first-chance exceptions."
There is no way to avoid the FCE's caused by HeapAgent/SmartHeap, short of disabling SmartHeap patching. They are much less prevalent in runtime SmartHeap. We understand the problem this creates if you are trying to debug your own exceptions. Note that NT itself uses excpetion handling and generates FCE's all the time, but the VC++ debugger doesn't stop on these since it knows that they are OS FCE's and hence that they are OK. Alas, VC doesn't treat SmartHeap FCEs the same as it treats the OS FCEs.
DWORD
"SkipDLLMessagesOn" to either "HKEY_LOCAL_MACHINE\Software\MicroQuill\HeapAgent\"
(to affect all apps) or "HKEY_LOCAL_MACHINE\Software\MicroQuill\HeapAgent\Apps\For earlier versions of SmartHeap and HeapAgent there is no way to automatically suppress this message box. HeapAgent allows you to suppress subsequent messages, and to disable patching on the specific DLLs, or on all DLLs, which will also keep the message from appearing. Both these fixes require either that the message appear at least once (so that you can choose to suppress it), or that you know in advance which DLLs you want HeapAgent to skip.
Some tips on using MemRegisterTask:
Note: HeapAgent 6 and some late (2000/2001) releases of HeapAgent 3.12 do not support stack checking. If your version of HeapAgent was released prior to the year 2000, read on for info on the limitations of this feature.
If you enable stack checking (compile with /Ge and turn on stack checking in the HeapAgent Settings dialog) HeapAgent will check for overflows of the stack frame. It will not check for overflows of individual local variables within the frame. So the error in this code:
Char str[5];
Strcpy(str, "abcdefg");
won't be detected by HeapAgent unless the overwrite is large enough to extend outside the stack frame. In this example, the compiler will allocate 8 bytes for local variable str for alignment reasons. There is no way to tell from an analysis of the stack that the source code shows an allocation of 5 bytes when 8 bytes were actually allocated. This is unlike malloc(5), where HeapAgent is performing the allocation at runtime so HeapAgent can see the size 5 being passed in even if the compiler CRT would have allocated 8 bytes. In the case of the stack, the number 5 exists only in source code, not in object code or at runtime).
If the default STL allocator, Allocator::Allocate, is implemented in terms
of operator new, it would automatically allocate via SmartHeap
if SmartHeap is defining new.
If you want an STL allocator that uses SmartHeap multiple pools, this would
be trivial to implement directly with public SmartHeap APIs: just pass a SmartHeap
pool as the "hint" parameter to Allocator::Allocate and have it call
MemAllocPtr().
Four possibilities:
mspdb60.dll. Try copying this file to a folder in your
system path.
GlobalAlloc, HeapAlloc,
etc) LoadLibrary
If either of the above applies to your app, try enabling system heap
patching. To do this, select your app in HeapAgent App Setup, click
the edit button and check the box Patch GlobalAlloc/LocalAlloc/HeapAlloc.
Enabling system heap patching will also cause HeapAgent to patch LoadLibrary.
If you try this and it causes other problems, uncheck the box Patch
GlobalAlloc/LocalAlloc/HeapAlloc and (if applicable) enable LoadLibrary
patching via the registry as explained here.
If you're not using HeapAgent, you can enable system heap patching via the registry. See your Getting Started Guide for details.
The only limitation that we're aware of with respect to using HeapAgent with NT services is if the service is started by the system before logon (and hence before the window system is up), the HeapAgent UI and error report dialog boxes are not available. In this case, you should uncheck the AutoLaunch HeapAgent User Interface box in HeapAgent App Setup and configure HeapAgent to log errors to a file.
Following are some suggestions for debugging DLLs loaded by inetinfo.exe:
To prevent autoloading:
'Out of Memory' error when trying to install HeapAgent -- This problem can
occur when your operating system is booting with the /3GB flag.
Fix is to remove the flag from boot.ini,
reboot, install HeapAgent, then restore the flag.
Always uninstall before installing a newer version, as previous versions of HeapAgent may cause problems for a new installation. If this problem is suspected, the fix is to uninstall HeapAgent, verify that HeapAgent references in the registry and HeapAgent files in the system folders have been removed, and then reinstall. The following procedure should accomplish this. This procedure can also be used to remove HeapAgent from your computer if the uninstall program misfires. NOTE: HeapAgent and SmartHeap share libraries and DLLs. If you are a software developer and have SmartHeap installed on your machine, debug SmartHeap may create registry entries referencing the HeapAgent DLL and/or HeapAgent libraries. Exercise care when editing the registry, and call MicroQuill tech support if you have any questions. You should not manually delete the HeapAgent DLL if you have SmartHeap installed.
Obligatory warning: Yikes! Please take care when editing the registry!
Note: This problem most likely will not occur in SmartHeap 6.02 and later.
Including smrtheap.hpp in an app on this platform may draw a compile error similar to the following:
Error 606: "smrtheap.hpp", line 138 #Missing exception specification; "void (*set_new_handler(void (*)()))()" was previously declared with an exception specification at ["/opt/aCC/include/new.h", line 52] _CRTIMP pnh MEM_ENTRY_ANSI set_new_handler(pnh) MEM_CPP_THROW;
The workaround is to remove the "&& defined(__LP64__)" from line 76 of smrtheap.hpp. smrtheap.hpp should be correct as is for either pre 11.0 HP-UX or 64-bit 11.0).On all platforms, if you get a compile error referencing the set_new_handler declaration in smrtheap.hpp, comment out that declaration. Usually that is
all that's required to fix this problem.
If after commenting out the declaration in smrtheap.hpp you
get a link error like the following, more drastic action may berequired.
/usr/ccs/bin/ld: Unsatisfied symbols:
exception::~exception() (first referenced in
SmartHeap/v6/hpux-11.00_acc33/lib/libsmartheapC_mtd.a(s
hnew.o)) (code)
typeid<exception> (first referenced in
SmartHeap/v6/hpux-11.00_acc33/lib/libsmartheapC_mtd.a(shnew.o
)) (data)
Class tables [Vftptr] dependent on key function:
"
exception::~exception()" (first referenced in Smar
tHeap/v6/hpux-11.00_acc33/lib/libsmartheapC_mtd.a(shnew.o)) (data)
What you may need to do is compile
the SmartHeap source files shnew.cpp and (in version 6.03 and later) shnewi.cpp,
and link to shnew.o and shnewi.o instead of libsmartheapC*.a. Contact MicroQuill
tech support for platform-specific instructions.
Note: Only SmartHeap ver. 3.x for UNIX offers separate mmap and sbrk libs.
The sbrk() approach grows the heap and the process address space in page increments as the sum of allocations and unallocated, fragmented heap space increases. Most tasks eventually reach some typical maximum heap footprint which remains constant with time. This technique is most efficient and is the default. The mmap() approach grows the heap and the process address space as required to contain all of the current allocations. Large unallocated blocks of the heap are returned to the OS for use elsewhere in the system.Ê Of course, some of the heap will remain unallocated and fragmented. This technique is less efficient, but is well suited to a few situations in which the sbrk() technique runs out of heap space prematurely. Our recommendation is to adopt the sbrk() approach for maximum flexibility and performance. If a problem is observed in your environment with excessive process address space, then you should consider trying the mmap() build to see if it helps. By default we ship the sbrk() version of SmartHeap. If you want to try the mmap() version, please contact us. The following text appears in the readme file included with most Unix versions of SmartHeap: ãThe version of SmartHeap on this install disk uses sbrk to allocate memory from the operating system.Ê In rare circumstances, you may discover that this uses excessive amounts of memory.Ê If you encounter this problem, please contact us and we'll send you a version of SmartHeap that uses mmap to allocate memory from the OS.Ê However, be aware that mmap is significantly slower than sbrk.”
In every process HeapAgent loads a tiny stub DLL that determines whether the actual HeapAgent DLL should be loaded in the current process. This enables HeapAgent to auto-load in applications by simply adding the app name to HeapAgent's setup utility. Unfortunately, the address used prevents Emacs from initializing.
There are several workarounds:
editbin /rebase:base=0x20000000 haloader.dll. The most convenient usage model for using MSDEV or MSVC with HeapAgent is to simply run the app under the debugger. HeapAgent will then load automatically with the app (provided the app is either specified in HeapAgent App Setup or linked with the HeapAgent lib) and you can debug normally under MSDEV/MSVC. Since the normal usage pattern under MSDEV/MSVC is to debug in the same session that is used to edit and compile, the spawn debugger option in the HeapAgent UI should rarely, if ever, be used. In other words, run the app under MSDEV/MSVC rather than running under the HeapAgent UI, because an existing instance of HeapAgent can connect to a running instance of an app whereas an existing instance of MSDEV/MSVC cannot.
The factory-default debugger command-line in HeapAgent 3.1? is "msdev -p%ld -e%ld". The command-line for VC++ 4.x/5.x is "msvc -p%ld -e%ld". If the command-line is changed for a particular app, that will continue to be the default for that particular app.
The named EXE or DLL isn't linked with Microsoft Visual C++ PDB debugging information, and it is statically linked instead of dynamically linked with the C Runtime Library. You must either link with debugging information or link with the C Runtime DLL. To specify PDB debugging information:
To specify the C Runtime DLL in Microsoft C++:
To use the Microsoft Foundation Class as a DLL:
SmartHeap is two libraries in one. There is the debug library designed to catch bugs in your code and a runtime library optimized for speed. The obvious difference between SmartHeap and HeapAgent is that the latter does not include a fast runtime library. SmartHeap and HeapAgent share the same debug library. Both offer a debug API that can be used to customize over two dozen different debug settings, format and redirect error reports, etc. HeapAgent accesses this API through a Graphic User Interface -- a feature not available in SmartHeap. For example, to change the number of guard bytes around an allocated block a SmartHeap user can insert in his code a call to the API dbgMemSetGuardSize. A HeapAgent user can accomplish the same end by simply opening the Debug menu and changing the Guard Size setting. No coding is required (HeapAgent users can also use the API directly if desired). The HeapAgent interface also includes a nifty collection of tools to aid in debugging: The allocation browser contains information (callstack, block size, et al) on all allocations monitored by HeapAgent. The source browser enables you to click a button and view the line of code that is linked to an error detected by HeapAgent. The dump browser is a raw memory dump which can be viewed in a variety of formats. The Agent utility can be programmed to automatically change debugging settings, set breakpoints or check for errors at strategic points in your application.
There is no way to dump the browser info to a file using the HeapAgent 3.12 user interface. Nor can you cut and paste the data in the browser. You can overcome this limitation programmatically: just call the HeapAgent API dbgMemWalkHeap and fprintf whatever details are desired to a file. If file/line or other debug info is desired, use dbgMemPtrInfo to get that information for each in-use allocation. For details, see the FAQ entry on obtaining callstack info.
HeapAgent 6 and later does not have this limitation.
The HeapAgent user interface is not ideally suited to displaying long callstack chains. You can increase the max number of chains to 16 in the Debug menu (under Settings). In an error report window or textual error log file you can scroll the length of the chain, but in the HeapAgent browser window your screen size/screen resolution is an absolute limit on how much you can see. Unfortunately there is no easy way to dump/paste browser data to a file or a spreadsheet. You can obtain callstack info programmatically with the API dbgMemPtrInfo or by using dbgMemSetEntryHandler/dbgMemSetExitHandler. These will fill the callstack field of DBGMEM_PTR_INFO and MEM_ERROR_INFO, respectively, with addresses of functions on the stack at the time the alloc was created and the time of the heap call, respectively. This isn't quite clear from the description of these API's in the Programmer's Guide, but see the definition of DBGMEM_PTR_INFO and MEM_ERROR_INFO in heapagnt.h.
HeapAgent 6 and later does not have this limitation.
A running application should show in the 'Attach running program' box if one of the following is true:
Each error detected by HeapAgent/SmartHeap is a discrete event, and there is no facility to relate one error to another. With HeapAgent you can filter leakage reports according to a variety of criteria: see the Find leakage... selection under the Debug menu. You can also view a list of all the errors HeapAgent has detected in the error browser (select Errors under the Browse menu).
With HeapAgent 6 and later, data in the error browser (and all the other browsers as well) can be exported to spreadsheet program for analysis.
It may also be possible to grab the raw data you want using the HeapAgent API, then manipulate and format it as desired. See some of the HeapAgent specific entries in this FAQ for additional info. See the HeapAgent User's Guide or the SmartHeap Programmer's Guide for additional info.
HeapAgent will indeed falsely report leaks from DLLs that terminate *after* the HeapAgent DLL has terminated, because HeapAgent leakage reporting occurs during HeapAgent DLL termination. The problem is that there is no way to control when the OS terminates dynamically loaded DLLs that are not explicitly unloaded.
You can easily suppress the leaks if that is an acceptable workaround: simply specify a module name of your DLL in the HeapAgent suppress dialog box and indicate error type of leakage. Alternatively, if you want HeapAgent to be guaranteed to terminate after your DLL and thus detect true leakage in the DLL, it is necessary to link your DLL with the HeapAgent library. The OS will then terminate the HeapAgent DLL after terminating your DLL rather than (potentially) vice versa.
Occasionally you may see "leaks" reported for allocations created in the CRT DLL, because the CRT DLL will generally terminate later than the HeapAgent DLL. Just suppress these leaks. Many CRT "leaks" are suppressed by default in HeapAgent.
Only HeapAgent can suppress error reports. If an app is linked to debug SmartHeap and a copy of HeapAgent is installed, HeapAgent can be used to suppress error reports in the SmartHeap app. Debug SmartHeap alone does not offer the option of selectively suppressing error reports (it is possible to suppress all errors -- see the relevant FAQ entry).
Generally the easiest way to suppress errors with HeapAgent is to click on the "Suppress" button in the error windows when the error is reported. This will take you to the error suppression dialog. By default the "Persistent" box in the lower right hand corner will be checked. Clicking OK will add this error to the list of errors that HeapAgent will not report for this app.
Note: Error suppression will take effect the next time you run the app. Error suppressions are app-specific. If for some reason you want to suppress a specific error in any application that you run under HeapAgent:
An optional way of suppressing leakage reports is to turn off the default leakage report for the termination agent, instrument your code with a few checkpoints, and define a new agent that checks for leakage between those checkpoints. Assuming you can modify the source code for debugging, that should not be very difficult. You will, however, need to take a little time to familiarize yourself with the procedure for using "agents". First time users of HeapAgent typically don't realize that HeapAgent only springs into action when your code executes some heap-related procedure. For example, if you set up checkpoints 3 and 4 and you want to check leakage between 3 and 4 on termination, you'll need to put a call to malloc, new, or some other heap function in you app just before it terminates. That will cause HeapAgent to check for any agents to apply, and it will run your leakage agent.
The Purify User's Guide devotes an entire chapter to the subject of using Rational Purify with custom memory managers (like SmartHeap), but the workarounds it describes basically involve modifying your code to create alternate execution paths, so that when the code is being purified the app uses CRT rather than the custom memory manager. In many cases this solution is less than ideal, but unfortunately we're not aware of any other workaround.
In most cases it is possible to purify an application that's linked to SmartHeap, but Purify will most likely report a lot (hundreds in some cases) of bogus errors (FMRs, FMWs, et al) because it misinterprets SmartHeap data structures. Purify gives you the option of suppressing certain catagories of errors, but in most cases this isn't a very good solution.
From our communications with Rational tech support about these issues, we understand
their position is as follows:
1) they do not recommend using Purify with custom memory managers, but
2) if you must use a custom mem manager, follow the instructions in their User
Guide for working with custom memory managers.
Two possibilities:
SmartHeap 4 specific: Apps linked with SmartHeap 4 may exhibit similar symptoms (mem usage constantly increasing over time even though resources are being freed). This problem will potentially occur with apps that create large numbers of threads, especially if the threads are dynamically created and destroyed. The solution is to turn off the small-block allocator. For important information on disabling the small-block allocator in SmartHeap 4, please click here.
SmartHeap-SMP 4 specific: In addition to MemPoolSetSmallBlockSize
it is necessary to call MemPoolSetFloor from each thread
in order for memory in default pools to be returned to the OS. The reason is
that SmartHeap/SMP has a default floor of 2GB in order to minimize system calls
(another speed/space tradeoff).
In SmartHeap-SMP as in SmartHeap 4 these APIs need to be called from each thread (MemDefaultPool is a thread-specific variable in SmartHeap-SMP). To set both the small block size and the floor size to 0 you would use the following code:
MemPoolSetSmallBlockSize(MemDefaultPool, 0);
MemPoolSetFloor(MemDefaultPool, 0);
These APIs may exact a small performance penalty. An alternative to setting
the floor to zero would be to choose a value between zero and infinity (such
as 1M) for the floor so that memory is returned but not every time a page becomes
free. Yet another alternative would be to call MemPoolShrink(MemDefaultPool)
immediately before terminating the current thread -- this would have the same
effect as changing the floor to zero. Setting small block size to zero is still
necessary. Yet another alternative would be to leave the floor at the default
of infinity and manually specify when memory in a pool is returned to the OS
with MemPoolSetFloor/MemPoolShrink at an appropriate time,
such as just before a thread terminates. Our recommendation would be to set
both the small block size and the floor size to 0. If performance degrades you
can consider the other options.
SmartHeap 5 specific: SmartHeap 5 (standard and SMP versions) does not return memory to the OS under any circumstances. The SmartHeap API calls described below affect how SmartHeap manages memory within the heap but has no effect on returning heap memory to the OS.
SmartHeap 5 features a new, more space efficient small block allocator, but
disabling it may still result in less memory consumption, depending on the alloc
pattern in a given app. Users who have upgraded to SmartHeap 5 and wish to continue
using the old SBA can do so by calling MemPoolSetSmallBlockAllocator
(see the SmartHeap 5 API reference for details).
The new SBA is more space efficient in part because it incurs less overhead and smaller granularity. Also, in a single-threaded app, when small blocks are freed they can be recycled for different block sizes. In multi-threaded apps and in SmartHeap-SMP, however, small blocks, once alloced and freed, can never be used for anything but subsequent allocs of the same block size. So for MT/SMP SmartHeap, the SBA is still suspect in some cases in apps whose footprints grow more than expected.
Another new API, MemPoolSetHighThreads(MemDefaultPool, FALSE)
will conserve space, but the extent of the impact on space very much depends
on the alloc pattern and thread concurrency of the app. If space is a problem,
you could try both MemPoolSetSmallBlockSize and MemPoolSetHighThreads
APIs to find the speed/space tradeoff that is right for a particular
app.
SmartHeap 6 & 7 specific: In SmartHeap 6.02 and above of the SmartHeap UNIX libraries, memory used for small block allocations is not returned to the operating system (OS) until the application terminates. This is a performance optimization, as munmap (used to return small block pages) is slow on most platforms. Small block memory that is no longer in-use is recycled, which further improves performance, since SmartHeap needs to make fewer expensive calls to the OS for more memory to manage.
Because of this design, memory or address space consumption may be higher than
expected. If this side-effect is considered unacceptable, SmartHeap 6.02 and
later offers an API, MemProcessUseMunmap, which
causes SmartHeap to aggressively use munmap to return small block pages to the
OS on those platforms that support it.
Allocation requests for large blocks are handled differently. SmartHeap 6 and
above defines a "large block threshold" value; blocks larger than
this value are allocated directly from the OS and freed directly to the OS.
This threshold value is 10 MB on most UNIX platforms, but 2 GB on HPUX and AIX.
A SmartHeap API, MemProcessSetLargeBlockThreshold,
can be used to modify this behavior.
Unlike SmartHeap UNIX, SmartHeap for Win32 does return free small block pages to the OS. The large block threshold on Win32 is 512 K.
On both UNIX and Win32, SmartHeap has a half-dozen or so APIs that can be used to fine-tune the speed vs. space trade-off. Please contact MicroQuill tech support for more info.
There are some tips for benchmarking performance in Appendix C of the SmartHeap Programmer's Guide. The most dramatic performance results would come from using multiple pools to improve application locality and thus avoid swapping, but this particular improvement will only be noticeable in a low-memory configuration typical of end-users but not typical of developers. It might be a good idea to conduct performance tests on target end-user configurations rather than on fast development machines.
For additional info on these issues, check out the performance doc on our tech support page.
Debug SmartHeap/HeapAgent requires specific compiler DLLs to read VC++ debugging info:
| VC++ 2.0 | dbi.dll |
| VC++ 4.0 | mspdb40.dll |
| VC++ 4.1/4.2 | mspdb41.dll |
| VC++ 5.0 | mspdb50.dll |
| VC++ 6.0 | mspdb60.dll |
| VC++ 7.0 | mspdb70.dll |
These DLLs are usually installed by default in the bin directory of their respective compilers. This error message is displayed if SmartHeap/HeapAgent is loaded for an app compiled using one of the above VC++ compilers and then run on a machine where neither that compiler nor its respective pdb file is installed. The message won't identify the specific DLL you need.
To fix the problem, obtain the correct pdb file and (if necessary) copy it to a directory (like winnt\system32) where HeapAgent can find it. Alternatively, you could rebuild the project with your current compiler installation.
Re Visual C++ 6: Some earlier releases of debug SmartHeap and HeapAgent 3.x may not be able to find mspdb60.dll. Copying this file to your system directory will work around this problem, or you can contact MicroQuill re an upgrade.
Re Visual Studio .NET: At this time (3/15/03) only SmartHeap version 7 and HeapAgent 7 support pdb info in VC++ 7. Contact MicroQuill tech support if you need a pre-release copy of version 7.
#include "shmfc.cpp"
or, if you're using SmartHeap/SMP:
#include "shmfcsmp.cpp"
For a non-MFC app, you add the source file smrtheap.c (SmartHeap/SMP users would add shsmp.c) to your project and specify the appropriate obj file in the Visual C++ link dialog. For example, for a release build using standard SmartHeap you would probably specify
.\release\smrtheap.obj
See the Getting Started guide for details. The main advantages of QuickStart are:
_SmartHeap_malloc
That should do it. (BTW, it's not a bad idea to force the reference anyway, as it helps insure proper linkage.).
Note: Your primary source of information on linking to Win32 SmartHeap is the SmartHeap Getting Started Guide for Win32. There is additional info here.
Note that in SmartHeap 3.0 and higher, malloc/MemAllocPtr are nearly as fast as MemAllocFS but without the initialization overhead.
HeapAgent 6 fully supports Microsoft Visual C++ versions 2.x, 4.x,
5.0 and 6, and Borland C++ versions 4.x (except 4.53 - see
below) and 5.x, and Borland C++ Builder 4.x/5.x/6.x. HeapAgent 3.12
supports Visual C++ and Borland C++, but not Borland Builder. Users of earlier
3.x versions of HeapAgent can upgrade to version 3.12 at no cost. HeapAgent
3.x users can upgrade to HeapAgent 6 for $99. Contact support@microquill.com
for more info.
SmartHeap version 6.0x for Win32 supports Microsoft Visual C++ versions 2.x, 4.x, 5.0 and 6.0, and Borland C++ versions 4.x (except 4.53 - see below) and 5.x, and Borland C++ 4.x/5.x. We have also tested and found no problems compiling and linking SmartHeap using the Intel C/C++ compiler for Win32 (an add-on for Visual C++).
SmartHeap 7 and HeapAgent 7 (to be released in late March or early April, 2003) will fully support unmanaged C/C++ builds with VC++ 7 (AKA Win2003). Most versions of SmartHeap 6 should work fine for release builds using this dev environment; the one known exception is projects that link to the MFC static lib. This latter issue is fixed only in SmartHeap ver. 6.03 and later. Debug builds are also supported, but SmartHeap 6.x is unable to access VC++ 7/Win2003 pdb info, and this will result in SmartHeap reporting addrsses rather than file and line numbers in its error reports. The workaround for this latter problem can be rather cumbersome for large projects, but it will be fixed in SmartHeap 7.
If you are using a compiler other than Visual C++, Borland C++, Borland C++ Builder or Intel C++: Most standard C/C++ compilers for Win32 are probably compatible with SmartHeap, given sufficient tweaking. However we have not done any testing with them and cannot provide technical support if you encounter problems.
Re. Borland 4.53: Borland 4.53 is actually Borland 4.51 plus a "CodeGuard" add-on from Borland. Borland 4.52 is actually an older release than Borland 4.53. Anyway, the "CodeGuard'ized" libs and DLLs in Borland 4.53 are mutually exclusive with SmartHeap libs and DLLs. SmartHeap will not work with that version of the Borland compiler.
MicroQuill cannot provide tech support if you are using HeapAgent with an unsupported compiler. We will try to help you if you run into problems, but that help will be limited since we are probably unfamiliar with the issues involved.
Note: Unsupported compilers may define new/delete with non-standard behavior. In that case, try the following:
_SmartHeap_malloc
unresolved external
symbol link error if MemInitDefaultPool
was called in a debug build with MEM_DEBUG defined.
The problem was in the header file smrtheap.h and
it was fixed in early June, '98. SmartHeap users who experience this problem with
version 4 should contact MicroQuill tech support for the fix.
dbgMemDeferFreeing to catch this type of bug.
For a short time there was a bug in debug SmartHeap 4 which prevented writing
errors to the log file. The log file was created but empty. If you experience
this problem please contact MicroQuill tech support for the fix.
No known problems with SmartHeap 5 or 6.
MemPoolShrink will compact pages of moveable
memory and free all pages with no allocations, so it is possible for the moveable
heap to be fluid and completely fragmentation free. However, this functionality
is only available for blocks allocated with SmartHeap's handle-based
API. Generally it is preferable to use the ANSI, pointer-based, or fixed-size
APIs. These three are easier to use (direct pointers to memory vs. handles) and
faster, because locking and memory movement are not occurring.
Calling MemPoolShrink for a pool with no moveable
pages will have no effect.
Registered users of SmartHeap 3.x/4.x/5.x can contact MicroQuill tech support and request a free copy of Smartheap 3.31, which will give info for operator delete.
CoTaskMemAlloc is implemented in terms of HeapAlloc.
Prior to version 4.0, SmartHeap replaced HeapAlloc (and the
other system heap functions LocalAlloc/GlobalAlloc) with its
own faster allocator on Win NT only (i.e., not Win 95). Beginning with SmartHeap
version 4.0 this patching is disabled by default. Developers on NT can
enable patching of HeapAlloc, etc, with a registry setting.
For details, see the SmartHeap 4.0 Getting Started guide entry on "SmartHeap's
automatic DLL patching". You can also access an html version of Getting
Started elsewhere in this knowledge base.
To sum up, you can use the registry to enable SmartHeap patching of HeapAlloc.
SmartHeap will then effectively replace CoTaskMemAlloc on
NT, but not on Win95/98. Otherwise SmartHeap will not interact with calls to CoTaskMemAlloc.
If you link your Solaris app to SmartHeap and don't see any change in performance,
you may need to juggle a bit with the linker library order. Use the debugger
or the Solaris system utility pstack to probe the application, and see whether
it's calling malloc() in libc or MemAllocPtr()
in SmartHeap.
Note: under Solaris a process can have any number of heap managers, but function symbols are bound (by default) at the process rather than at the EXE/DLL level, meaning that any references to malloc in the process will be bound to the first malloc definition that was loaded (which is the one referenced in the EXE). It is possible to build a DLL in Solaris such that it uses its private malloc defiition rather than the process definition by using a linker flag.
// define an object
class B
{
public:
~B();
};
B::~B()
{
printf ("B::~B()\n");
}
// somewhere in your code...
B * pB = new B[5];
MemCheckPtr (MemDefaultPool, pB);
With an array of objects, the object pointer is usually not equivalent to the value returned by operator new -- it is an offset therein, since the compiler stores the array size at the beginning of the memory block returned by the allocator. The code above is effectively doing the following:
B * pB = new B[5];
MemCheckPtr(MemDefaultPool, &pB[1]);
which will fail in Debug SmartHeap as &pB[1] is not a pointer value returned by SmartHeap (pB is). Runtime SmartHeap does not detect the error because its checking is not as extensive as debug SmartHeap's (runtime SmartHeap detects that the pointer is part of a heap block in the indicated pool, but does not perform the slower scan of heap blocks to see if this matches the beginning vs an offset into a block).
There is another debug-only API in SmartHeap that will validate a pointer
that is an offset into a heap block if that is what is desired: dbgMemCheckPtr.
MemPoolSetPageSize. If an app
has its page size set to 64 K, then one small alloc would result in the pool consuming
about 70 K.
A SmartHeap page is the smallest increment of memory that can be added to or released from the pool. In order to release a page of memory from the pool, every allocation made using that page must have been freed. Thus a 64 K page with a 10 byte allocation will be held by the pool as long as the alloc is in use.
Each SmartHeap platform has a default page size. For details, see the section on "Page tuning" at the back of the SmartHeap Getting Started guide.
MemPoolSetSmallBlockSize (see the
Programmer's Guide and other entries in this FAQ for more info).
MemFreeDefaultPool under MemDefaultPool
in the SmartHeap Programmer's Guide. It should generally not be used in
apps where SmartHeap replaces malloc or new
as it would free blocks in use within the CRT. A new default pool will
be created on a call to malloc or new
subsequent to MemFreeDefaultPool.
shdw32m(t).lib or shlw32m(t).lib) crashed with the error message
Debug Assertion Failed!
program xxxx.exe
File: dbgheap.c
Line: 1011
Expression: _CrtlsValidHeapPointer(pUserData)
This error can only occur if Debug C runtime library or Debug MFC is used
-- the file dbgheap.c is part of the debug VC CRT, as is
the API CrtIsValidHeapPointer. So the invocation of this
error in association with the Runtime SmartHeap libraries indicates a runtime/debug
mismatch. Runtime SmartHeap does not support debug versions of the C runtime
libraries. If your application links with the Debug C runtime library it must
also link with the Debug SmartHeap library.
dbgMemCheckPtr(ptr, MEM_POINTER_HEAP, size)
will check to see if ptr to ptr+size falls
within a valid allocation. Enumerating the entire heap is the only way to perform
the check in Runtime SmartHeap.
MemSetErrorHandler with a NULL
parameter. All SmartHeap APIs will then silently return error return values on
failure.
AfxDoForAllObjects
and the CMemoryState member functions. For the unsupported
APIs, SmartHeap defines stubs that print error messages that point to the corresponding
SmartHeap APIs offering similar functionality. The solution is to use the specificed
SmartHeap function instead.
Since these MFC diagnostic classes are only available in debug builds (_DEBUG
defined) this is not an issue for release builds.
| "Visual C++ 4.0 and higher includes debugging versions of the C runtime libraries. Debug SmartHeap supports these libraries, but Runtime SmartHeap does not. Therefore, if your application links with the Debug C runtime library, it must also link with the Debug SmartHeap library (haw32m.lib) rather than the Runtime SmartHeap library (shlw32m[t].lib or shdw32m[t].lib). Debug SmartHeap overrides the Visual C++ functions _calloc_dbg, _realloc_dbg, _free_dbg, _msize_dbg, _expand_dbg, _CrtCheckMemory, _CrtDumpMemoryLeaks, _CrtIsValidHeapPointer, _CrtIsMemoryBlock, _CrtSetAllocHook, and _CrtSetBreakAlloc. Debug SmartHeap emulates the behavior of each of these Visual C++ routines, so you can use these routines interchangeably with Debug SmartHeap APIs." |
MemSetPatching has a couple of limitations:
MemSetPatching works if exported from an EXE, and should be
considered an alternative to manually editing the registry.
For more info on SmartHeap's patching, click here!
Note: Only the SmartHeap DLL does any patching of DLLs. Debug SmartHeap always uses the DLL, but with release builds you have a choice. See our tech support page for more info on SmartHeap patching.
realloc should be released with free,
even if the block was originally allocated with new
Note: If you're using Visual C++ or Borland C++, please see the entries
in the FAQ specific to those compilers.
There is nothing platform- or compiler-specific about the file/line reporting
mechanism in Debug SmartHeap. It is simply using ANSI C macros to pass file/line
to the heap routines.
Several things can prevent SmartHeap from reporting file/line info, including bugs in your code. For example, an overwrite can scribble on stack parameters or mangle the heap such that file/line info is erased. If the problem is not caused by a bug, the following test case should help you run it down:
#include <stdio.h>
#include <stdlib.h>
#include "heapagnt.h"
void main()
{
char *x = new char;
char *y = malloc(1);
delete x;
delete x;
free(y);
free(y);
}
This code should report double-free errors and show file/line for the allocs and the frees. If file/line is missing, try the following:
new(__FILE__, __LINE__).new is being
linked. operator new(char *file, int line, size_t size).
SmartHeap should work fine in a COM project. Note, however, that in Win32 many
COM functions call HeapAlloc for memory. SmartHeap version
4.x and later does not replace HeapAlloc(SmartHeap
3.3 and 3.4 do replace HeapAlloc on NT). If you want
your 4.x or later version of SmartHeap to replace the system heap functions
(GlobalAlloc, HeapAlloc, etc) you can
enable patching of HeapAlloc via a registry setting: see
the section "SmartHeap's automatic DLL patching" in our Win32 Getting
Started guides for details.
Note: using the SmartHeap versions of GlobalAlloc/HeapAlloc/etc
may cause problems for your application, because Microsoft system DLLs make
undocumented calls to the system heap functions which the SmartHeap replacements
for these functions are unable to handle. We generally would not recommend using
SmartHeap's HeapAlloc replacement in complex software projects.
Even if SmartHeap is not replacing HeapAlloc, multi-threaded apps that use
HeapAlloc intensively (again, COM projects are a good example)
may benefit from our HeapAlloc wrapper. The HeapAlloc wrapper doesn't contain
SmartHeap or implement heap; it just patches HeapAlloc and
GlobalAlloc and re-directs these call to per-thread HeapAlloc
heaps. Usage requires just one LoadLibrary call. If your app runs on SMP boxes
this design can significantly reduce contention in the system heap.
SmartHeap has no upper limit on how much memory it can manage, so the limit will be that set by the OS. Typically, though, the allocation process wastes some address space, and a process will run out of address space before (sometimes well before) the heap manager has stored 2 GB of data/objects. At that point you get an Out of Memory error. It's important to remember that wasted address space does not equate to wasted physical memory.
If you have an extremely large memory pool (more than 128 MB), or if your application is so large (hundreds of megabytes) that address space is short, you should set the SmartHeap page size to 64K. This will minimize wastage of address space. The default SmartHeap page size in Win32 is 16 K. See the SmartHeap Programmer's Guide and other entries in this FAQ for additional info. Note that the page size is a per pool value,
Note: Beginning with SmartHeap version 5 the default page size is 64 K on all platforms.
If the 3 GB option is enabled on NT Enterprise, SmartHeap can access it.
To have SmartHeap override malloc in a particular source file, just include
the SmartHeap header shmalloc.h. All calls to malloc/realloc/calloc/free
will then be defined as macros that map to (for example) MemAllocPtr(MemDefaultPool).
If for some reason you do not want SmartHeap to replace malloc
and/or new, you can remove the appropriate module(s) from
a SmartHeap static library (shared/dynamic libs must be rebuilt in order
to remove functions). Note that if your project links to the SmartHeap static
lib, it is usually adviseable to link CRT statically as well.
To prevent SmartHeap from overriding malloc remove shmalloc.obj
and shmalimp.obj from shlw32m(t).lib (or shlsmpmt.lib
if using SmartHeap/SMP):
lib shlw32m(t).lib /remove:shmalloc.obj
If you remove the shmalloc object module from a SmartHeap library, you will need to define SmartHeap_malloc at file scope. For example:
int SmartHeap_malloc = 0;
To prevent SmartHeap for Windows from overriding new, remove
shnew.obj, shnewimp.obj and shnewvc.obj from shlw32m(t).lib
(or shlsmpmt.lib, if using SmartHeap/SMP)
lib shlw32m(t).lib /remove:shnew.obj
There is no static lib for debug SmartHeap. If you don't want debug SmartHeap to define malloc/new, do the following:
lib haw32m.lib /remove:shnew.obj /remove:shnewjmp.obj...etc
extern int SmartHeap_malloc = 0;See also the section "Compiler functions that SmartHeap overrides" in the SmartHeap Win32 Getting Started Guide.
On UNIX platforms the procedure is very similar to Win32, only the commands
and the file names are different. To prevent SmartHeap from overriding malloc
with the statically linked non-debug SmartHeap library:
ar d libsmartheap.a shmalloc.o
If you remove the shmalloc.o object module from a SmartHeap
library, you'll need to define SmartHeap_malloc at file
scope. For example:
int SmartHeap_malloc = 0;
To prevent SmartHeap from replacing new, don't link to
the C++ archive lib (libsmartheapC*.a) or (if
you're using SmartHeap 5 or earlier) remove the object module shnew.o.
Every SmartHeap pool can manage both fixed and variable-sized blocks. There
are (possibly empty) sub-heaps within each pool for different block types. The
only difference between MemPoolInit and MemPoolInitFS
is that the latter specifies at initialization the FS block size whereas the
former initializes with the default FS block size; the FS block size can be
changed in either case (prior to the first call to MemAllocFS)
with MemPoolSetBlockSizeFS.
if you use only one block type for a given pool, no memory or other overhead will be incurred for the other block types. You can choose to use separate pools for different block types or use the same pool --- it is a matter of programmer preference.
There are other sources of resource contention in addition to CRT heap. All the resource contention must be eliminated in order to significantly improve CPU utilization (i.e. if all threads need to concurrently access resources A, B, and C, making resource A (heap) available to all threads concurrently is necessary but not sufficient to solve the problem).
SmartHeap and SmartHeap/SMP 4 and 5 offer limited support for pre-SPARC 9 systems. This support requires that the application link to the SmartHeap shared library. In order to provide this support and still give good performance with SPARC 9 a SPARC 9 specific optimization was placed in a "filter" shared lib, libatomic_ultra.so. It is necessary to distribute both the SmartHeap shared lib and libatomic_ultra.so with your app to ensure that both SPARC 9 and pre-SPARC 9 systems can take advantage of SmartHeap's performance. Here's the text from the SmartHeap Getting Started Guide:
The multi-threaded version of SmartHeap implements a serialization optimization that employs a SPARC 9 instruction. In order to support pre-SPARC 9 targets, we have placed the optimization in a "filter" shared library, libatomic_ultra.so. If libatomic_ultra.so is not present at runtime, or if libsmartheap_mt.so is loaded on a pre-SPARC 9 system, SmartHeap will not use the optimization. There is no functional difference with or without libatomic_ultra.so; it is just a performance optimization that impacts only highly threaded applications.
SmartHeap 5.01 and 6.00 do not support pre-SPARC 9 systems, and libatomic_ultra.so is not used in these versions.
Note: If you statically link with SmartHeap your app will run only on SPARC 9 systems.
Debug builds that define DEFINE_NEW_MACRO and compiled with the SunWorkshop C++ compiler 4.2 (and possibly others) draw a compile error when the source code uses operator new with arrays of objects. A typical error message is "libC: delete[]' does not correspond to any 'new[]". This appears to be a compiler bug as the problem occurs whether or not SmartHeap is present. The following program will exhibits the same error (compiled with simply "CC test.C"):
class CUseless { public: CUseless() { } ~CUseless() { } };
void *operator new(size_t size, int parm);
void *operator new(size_t size, int parm) { return new char[size]; }
void main() { CUseless *things; things = new (1) CUseless[10]; }
This compiler does not support placement syntax for operator new with arrays of objects. Consequently, DEFINE_NEW_MACRO cannot be used with these Sun C++ compilers, because DEFINE_NEW_MACRO uses the placement syntax to pass parms to operator new.
This can happen when an EXE or DLL is linked to one version of SmartHeap and another (perhaps 3rd party) DLL is linked to a different version. This will generate a SmartHeap error message when the app tries to load. The reason two different versions of the SmartHeap DLL cannot be used in the same process is that they will both try to patch CRT/MFC DLLs: clearly, only one version of SH can patch the CRT DLL in a given process and there is no way SmartHeap can know which version should patch.
There are three ways to solve this:
Note: The HeapAlloc wrapper DLL (shsmpsys.dll) has no effect on uni-processor systems: only on SMP systems that make a lot of calls to HeapAlloc with the default process heap from multiple threads.
shsmpsys.dll should be loaded with LoadLibrary, either from the EXE or from another DLL. There is no import lib for this shsmpsys.dll because it has no exports -- it simply patches HeapAlloc, etc. in the current process. It should work regardless of when it is loaded, but the earlier the better. When shsmpsys.dll is loaded, it does a LoadLibrary on itself to prevent it from being unloaded. Thus it will remain loaded until process termination, even if it was loaded by another dynamically loaded DLL.
We have heard from some users that apps linked to SmartHeap and built with Borland C++ 5.02 may fail to catch exceptions that ought to be caught. This may be due to the fact that BC++ changed its exception handling mechanism between versions 5.01 and 5.02, and SmartHeap 3.x and 4.x for Borland were built with 5.01.
If you believe this is a problem with your app, here is a workaround:
As is often the case with SmartHeap, the considerations are different in Win32 and UNIX.
It is generally OK to link multiple targets (several DLLs, for example) to the SmartHeap static lib, as long as you use the same version of SmartHeap with each target. A DLL statically linked to SmartHeap has a private copy of all of SmartHeap and does not even know if other DLLs or the EXE in the process are using SmartHeap. However, even if one DLL is freeing memory allocated by the other, SmartHeap will find the pool from the pointer passed in (not from a global variable), so there should be no problem.
Note: It is generally not adviseable to use the SmartHeap static libs if other modules in your app are using CRT or MFC in a DLL. See the SmartHeap Getting Started Guide for Win32 (the section "SmartHeap's Automatic DLL Patching") for more info. Also see the doc on SmartHeap patching on the tech support page of this web site.
It is OK to link multiple targets to the SmartHeap dynamic lib, as long as all targets use the same version of the lib (i.e., they must all call the same SmartHeap DLL). However, because the SmartHeap DLL patches CRT and MFC DLLs, it is generally unnecessary to do so. In other words, if you have an EXE and a lot of DLLs, and you want SmartHeap to manage memory for the entire process, simply link your EXE (or a single DLL) to the SmartHeap dynamic lib (either shdw32m(t).lib or shdsmpmt.lib). When your EXE loads SmartHeap will patch CRT/MFC heap functions and henceforth all DLLs loaded by the process will use SmartHeap automatically. Please see the Getting Started Guide and the SmartHeap patching doc on the tech support page of this web site for additional important info
Note: In SmartHeap 4.x and 5.0 every target linked to the SmartHeap DLL got its own default pool. If one of these DLLs was unloaded before the process terminated it's default pool would persist. If the same DLL was later reloaded, a new default pool would be created. In some applications (web servers, for instance) where a DLL could be unloaded/reloaded thousands of times, this behavior could create a very significant memory leak. Beginning in SmartHeap 5.01, all the modules linked to the SmartHeap DLL share a single default pool.
It is generally OK to link multiple targets to either the SmartHeap static or dynamic libs, as long as you use the same version of SmartHeap with each target. See the "Platform notes" section of your Getting Started Guide for info on any platform-specific issues.
The HeapAgent Total Memory used by application figure is obtained by enumerating all virtual objects in the target process and summing all regions with a MEM_COMMIT attribute. This is the total amount of memory currently allocated by the process.
The figures in task man are VM Size (same a perf mon private bytes): this is the total non-shared memory in the process, which includes all heap (except SmartHeap shared pools) and other non-shared data. Code is shared so it isn't included in this figure. Neither is other shared mem. The other figure in task man is "Mem Usage", which is physical memory currently allocated to the process (i.e. RAM). The other figure in perf mon is "virtual bytes" which is address space, which isn't memory at all. This number should match the Address space used by application in the HeapAgent statistics browser.
If your MFC project does not use precompiled headers, here's how to use the 'Quick Start' linking procedure:
./release/shmfc.obj (./release/shmfcsmp.cpp
for SmartHeap/SMP) as the first entry on the Object/library modules line in
the link dialog.If you install a full version of HeapAgent (or debug SmartHeap) but cannot run it because you get a message saying the the HeapAgent trial has expired, do the following:
SmartHeap 5 and SmartHeap 6 use different locking mechanisms and there is no way to make SH6 behave like SH5 in this regard. However, the large-block allocator is essentially the same except that it returns memory to the OS by default in SH6, which can have a performance impact with some allocation patterns.
To closely emulate SH 5 in SH6 we suggest the following settings:
GDI functions may faile when operating on a large block allocated by SmartHeap,
when that block straddles two adjacent VirtualAlloc ranges.
Say, for example, you're creating a bunch of 300K bitmaps. SmartHeap gets 1MB
from VirtualAlloc and allocates 3 of your
300K objects out of that. For the fourth object it allocates another 1MB, and
this object will straddle those two 1MB ranges. The data for the
fourth object is written into the pointer block with no problem, but if you
call a GDI function on the block it may fail.
We suspect this may be a bug in the GDI function. Most likely, the GDI routine
is using VirtualQuery to validate the pointer, but it is
failing to take into account the adjacent VirtualAlloc object.
It is not related to physical memory mapping as virtual memory is mapped to
physical memory in non-contiguous 4K units -- VirtualAlloc
objects do not represent contiguous blocks of physical memory.
There are several ways to work around this problem:
MemProcessSetLargeBlockThreshold
to set the large block threshold (512 KB by default) to some size smaller
than the block sizes that are causing the problem. Blocks larger than
the large block threshold will be managed as direct VirtualAlloc allocations.MemProcessSetLargeBlockThreshold(64K), which will
ensure that all allocs will be within contiguous VirtualAlloc
objects; MemProcessGrowIncrement so
that the entire SH heap is within the same VirtualAlloc
block. If the app MemProcessGrowIncrement during
initialization, specifying a value that equals or exceeds the peak heap usage
for the app. As long as there is sufficient contiguous address space for the
entire heap, this would guarantee that all portions of each alloc would lie
within the same VirtualAlloc object, except for the 1MB region that will exist
before MemProcessGrowIncrement was called. Versions of Win32 SmartHeap earlier than version 6.02 treated __stdcall CObject::delete(void *, void *) (which was introduced in VC++ 6 MFC) incorrectly. This could result in link errors similar to this:
shdsmpmt.lib(shmaljmp.obj) : error LNK2005: "public: static void __stdcall CObject::operator delete(void *,void *)" (??3CObject@@SGXPAX0@Z) already defined in XXX.lib(YYY.obj)
All SmartHeap Win32 libs built on 11/1/2001 (ver. 6.02) or later contain the fix for this problem.
In SmartHeap 6.02 the type of the size parameters and return values to all SmartHeap APIs changed from unsigned long to size_t. This was necessitated by Win64, where size_t is larger than unsigned long.
size_t and unsigned long are the same size in Win32 so a recompile isn't generally necessary, except that SmartHeap-specific overloaded operator new variants that took unsigned long now take size_t and the mangled names are different. So if you see the following link error you will need to recompile source files that include smrtheap.hpp:
tcall.obj : error LNK2001: unresolved external symbol "void * __cdecl shi_New(unsigned long,unsigned int,struct _SHI_Pool *)" (?shi_New@@YAPAXKIPAU_SHI_Pool@@@Z)
Whereas SmartHeap 5.0x never returned memory to the OS until process termination, SmartHeap 6 introduced three new APIs which give the programmer the ability to control when memory is returned and how much is returned. The default settings strike a balance between memory footprint and performance (i.e., speed) that will be suitable for most applications.
In most cases SmartHeap 6.0x also provides better absolute speed than SmartHeap 5.x.
Win32 SmartHeap 6.0x includes support for Borland C++ Builder.
What changed from
SmartHeap 6.00 to 6.02
The main difference for Win32 is further space optimizations, with the page size
going back to 64K (16K in SH 6.00). The small blockallocator uses 4K pages that
grow dynamically to 64K. The result is that the small block allocator uses less
memory, larger blocks can be sub-allocated in the medium block allocator, and
largeblocks use less memory because their size is rounded to a 4K rather than
64K multiple.
SmartHeap/SMP scaling is improved, particularly on higher processor counts -- SH 6.02 scales linearly up to 64 processors.
Performance of the SmartHeap multithreaded runtime libs is improved.
The handle-based API (MemAlloc) is no longer supported in SmartHeap/SMP, which makes some further SMP speed optimizations possible.
There is also a new API: MemAllocAligned, introduced primarily for the benefit of UNIX valloc and memalign.
Several new "tuning APIs" have been added to give additional controls over when memory is returned to the OS and how much is returned.
On Win32, SmartHeap supports the Visual Studio .NET compiler in unmanaged C/C++ projects. However, the new pdb format is not supported, so
And there are a number of minor bug fixes.
What changed from SmartHeap 6.02 to 6.03
The main difference for Win32 is improved support for VC++ 7. The known linking problems have now been resolved. There is also a new API, MemProcessCoelesceSystemAllocs, to prevent SmartHeap from coelesceing blocks obtained from the OS in order to satisfy requests for large blocks of memory. This API was developed specifically to work around a bug in the Microsoft GDI.
SmartHeap UNIX has additional space optimizations. More memory and address space is returned to the OS when no longer in use, and there is a new API (MemProcessUseMunmap) to aggressively return address space on Solaris, DEC UNIX and SGI.
On both Win32 and UNIX the default sub-alloc threshold was raised from 16K to a little under 64K. This design should use less memory in most cases, and particularly when a lot of blocks in the range of 16K to 64K are allocated. A new API (MemPoolSetMaxSubAlloc) lets the user tune the suballoc threshold when appropriate.
And there are a number of minor bug fixes.
What changed from
SmartHeap 6.03 to 7.00:
The noteable changes are in Win32 and Win64. All the known compatability problems
with the Visual Studio .NET/VC++ 7 have been fixed. Debug SmartHeap now obtains
file and line numbers for error reports transparently from the debug info generated
by Microsoft compilers. Multithreaded SmartHeap and SmartHeap/SMP both support
Intel's new hyperthreading technology.
On UNIX, SmartHeap for linux also supports hyperthreading.
And there are a few minor bug fixes.
SmartHeap does not throw exceptions. If your application is linked to SmartHeap and you want it to throw exceptions when SmartHeap detects a heap error you should write an error-handling routine and call the SmartHeap API MemSetErrorHandler to establish it. From this custom error handler you can throw the desired exception.
For more information on how to customize error handling see the reference on MemSetErrorHandler in Chapter 4 of the SmartHeap Programmer's Guide.
We get asked this question occasionally. Multithreaded SmartHeap can easily handle a situation where one thread frees memory that was allocated by a different thread. There's no performance penalty involved.
SmartHeap 6 & 7 don't support moveable memory. If you link to SmartHeap 6 or 7 and call a moveable mem API (MemAlloc, for example) you will get a MEM_UNSUPPORTED error.
If you don't need moveable memory we would suggest that you use MemAllocPtr rather than MemAlloc. MemAllocPtr provides faster allocations and eliminates the need for you to deal with memory handles.
SmartHeap allocates its pages (and large blocks) on 64K boundaries.
The pointers SmartHeap returns to the user are not on 64K boundaries because SmartHeap stores
header info in the first few bytes of pages and large blocks.
SmartHeap 6 includes an API, MemAllocAligned, which will return memory aligned
on any boundary desired. SmartHeap also defines the UNIX equivalents valloc and
memalign, and on Win32 the VC++ 7 CRT equivalents _aligned_malloc and _aligned_offset_malloc.
The ANSI standard specifies that malloc and new return memory suitably aligned for any object, but SmartHeap versions 5.x, 6.x, 7.01, and 7.1 potentially return non-aligned memory in order to conserve space. Specifically, SmartHeap’s small block allocator (SBA) aligns on a word multiple. The first SBA block in a page is aligned on a double boundary, but subsequent blocks are aligned on a word multiple. So on a 32-bit architecture, a 4 byte SBA request will return 4-byte aligned, but an 8 byte request will return 8 byte aligned memory. Typically, compilers pad structures that contain doubles to a double multiple, so doubles will generally be suitably aligned even with SBA. However, this isn't guaranteed.
In order to guarantee sizeof(double) alignment, you must disable the SBA.
SmartHeap’s medium and large blocks always align on double boundaries.
To disable the SBA use the following call:
#include <smrtheap.h> MemPoolSetSmallBlockAllocator(MemDefaultPool,
MEM_POOL_SMALL_BLOCK_NONE);
Note: Beginning with version 7.2, SmartHeap returns memory aligned as per the ANSI standard..
Lines of source code In SmartHeap version 7:
Aggregate SmartHeap source is a sum of all the above numbers, or ~134K SmartHeap + 75K HeapAgent.
If your version of the Windows OS supports the /3GB boot option, SmartHeap
will be able to us the extra space.
Note: This information applies to SmartHeap/SMP only.
The SmartHeap/SMP runtime heap is designed so that multiple threads can perform heap operations concurrently.This can lead to significant performance improvements, but it can also expose subtle timing bugs in your code; bugs that would never manifest in a conventional heap design, where thread access is serialized. Because debug SmartHeap serializes thread access as well, it likely won't detect these bugs. For the same reason, if you remove SmartHeap/SMP from your runtime build, the problem will probably go away.
To address this, all of our SmartHeap/SMP distributions include a special set of debug-smp libraries, designed to be used in cases where all the following are true:
The debug-smp libs are hybrids: like the runtime smp libs they allow concurrent
thread access to the heap; like the debug libs they have some bug detection
code. Use them as you would the SmartHeap runtime libs. UNIX users should not define MEM_DEBUG or DEFINE_NEW_MACRO on
the compiler command line when using these libs.
naming conventions are different on Windows and UNIX. On Windows, the specialized smp debug libs are named sh?smpmtd.lib, where '?' is either "d" ( the dynamic lib) or "l" ('el', the static lib). On UNIX they are libsmartheap*smpd.*
For general purpose debugging, use the regular SmartHeap debug libs. They detect many more error conditions, and they provide more information for easier debugging.
For more info on this topic, see the SmartHeap/SMP Getting Started Guide for your platform, and email questions to support@microquill.com.
If you try to link SmartHeap 6 or later to your project with an older Borland tool (Borland C++ 5.x or early versions of C++ Builder) you may see a linker error about a missing library. For example:
Error: Unable to open file 'STLP.LIB'
This is because we build the SmartHeap for Borland libs with C++ Builder 6, and Builder 6 uses these libs. If you encounter this problem, contact MicroQuill tech support.
The main cause of fragmentation is large requests that cannot be satisfied
due to heap space being splintered into small free spaces.
if you have 1MB or greater blocks and you want to avoid fragmenting the entire
address space with blocks smaller than this, SmartHeap may help. If the SmartHeap
large block threshold (LBT) is set to 1MB then these large requests will occur
top-down from high in the address space while all smaller requests will be
handled bottom-up from low in the address space. The space in the middle is
thus retained free and the high address space is never splintered by small
blocks. (many operating systems, including Microsoft Windows, allocate
everything starting low in the address space and the entire address space can
become fragmented
with
interleaved
large/small
requests).
One caveat is that if there are many > 1MB blocks of different sizes that come and go, then the high address space can become fragmented. Tuning the LBT value for optimum space utilization would help here and this is very app-specific.
RE the LBT: this value is 512K by default in Microsoft Windows, and 10 MB on most UNIX platforms. It can be changed by calling the SmartHeap API MemProcessSetLargeBlockThreshold.
By default, blocks larger than SmartHeap's "large block threshold" (512K on Windows, 10MB on UNIX) are allocated by the OS and freed directly to the OS, though SmartHeap controls their location in the process's address space. You can change the large block threshold by calling the SmartHeap API MemProcessSetLargeBlockThreshold.
Larger values for the large block threshold generally result in better performance but potentially larger process footprint
A Dynamically loaded DLL that links to a SmartHeap static library may appear to "leak" memory if both of the following conditions are met:
What happens is, when the DLL unloads the SmartHeap default pool is not destroyed (because some allocs may still be in use), and when the DLL is reloaded SmartHeap creates a *new* default pool. This legacy of old default pools constitutes the leak.
To avoid this problem do one of the following:
dllexit.c to your DLL project source. Contact
MicroQuill tech support (support@microquill.com)
to obtain a copy of dllexit.c.Note that the above procedures were developed in May, 2003. If your SmartHeap
libraries were released before that date, contact MicroQuill tech support for
updated Quick Start files and/or dllexit.c.