Are you talking about the size of the working set or the heap memory? Heap memory is simple: debugging VC ++ Runtime has _CrtSetAllocHook, which calls a function provided by the user with each allocation / reallocation / free memory call.
http://msdn.microsoft.com/en-us/library/820k4tb8.aspx
You can set the hook and then summarize the memory allocation. If you hit your threshold, you can call _debugbreak () to go to the debugger.
source share