Questions about SetWindowsHookEx () and Connectivity

Question

Here is some background information. I am working on a dll replacement that was used in the dll injection method using the AppInit_DLLs registry entry. His goal was to be present in every process and set intercepts in GDI32.dll to collect print information. It's a kind of funky way to get what we want. The DLL itself is over 10 years old (written in Visual Studio 97), and we would like to replace it with something less invasive than the injection dll.

It seems that SetWindowsHookEx() might be what we are looking for. I had some problems with this, but I also had some discussions with colleagues about whether this tree is worth barking. Here are some questions that we could not determine:

• When we connect a routine from dll, for example StartDoc() from GDI32.dll, do we really get a notification every time any other process uses this property from this DLL? This is the kind of functionality that we received with our .dll injection, and we need the same functionality in the future.

• When the hook is started, is the hook processing procedure performed in the process space of the process that initiated the actual call, or in the process space of the process that installed the hook? My opinion is that it should run in the process space of a process called a subroutine. For example, if a program calls StartDoc() from GDI32.dll, it will process the hook processing code "entered" into its space and executed. Otherwise, there must be some kind of interprocess communication that is automatically configured between the calling process and the process that sets the hook, and I just don't see it in this. In addition, it is necessary that this interception processing procedure be performed in the process space of the calling process, since one of the things he needs to know is the name of this calling process, and I'm not sure how to get this information if it actually did not work in this process.

• If the hook processing procedure is written using a managed .NET environment, will it break when connected to a process that does not use a managed .NET environment? We would really like to move away from C ++ here and use C #, but what happens if our hook receives a call from a process that is not controlled? As stated earlier, I think our hook processing routine will be executed in a process that was originally called the subroutine that was connected. But if that is the case, then I would have thought that we would have problems if this process does not use the .NET runtime, but the incoming processed code.

Answer 1

• Yes.

• This is usually the first one: it runs in the context of the process whose event it connects to.

  After a successful call to SetWindowsHookEx operating system automatically adds a DLL hook (the one that contains the callback function) to the address space of all target processes that meet the requirements for the specified type of binding. (Of course, the interception code is not necessarily entered immediately.)

  An exception to this general rule are low-level keyboards and mouse hooks ( WH_LL_KEYBOARD and WH_LL_MOUSE ). Since these types of hooks are not injected into client processes, the callback is called in the same thread that was originally called SetWindowsHookEx .

• This last point is important to keep in mind to answer your third question. Because low-level keyboard and mouse hooks are the only two global hooks that do not require DLL injection, they are also the only two types of hooks that can be written in .NET managed code.

  For other types of hooks, your concerns expressed in the question are accurate. You will need to write these DLL files with hooks in C or C ++. Of course, the rest of your applications can be written in a managed language. The only thing that matters is the dll file.

You might consider looking for Microsoft Detours or EasyHook .