Writing a Windows NT Subsystem

Question

I would like to try to write my minimal NT subsystem in Windows 7 for purely educational purposes - something like the equivalent of posix.exe barex bones in the Microsoft subsystem for Unix applications.

But I can not find public documentation on this topic. What API do you need to implement the subsystem? How is it registered in Windows? How to create an image of a subsystem (which flags should be set in the PE header, etc.)?

Most of all I would like to find a book or website with an overview of the whole subject or even the source code for the NT subsystem "hello world" that someone wrote. But everything would be appreciated if you can point me in the right direction here ...

Answer 1

Here are the main components of the subsystem:

• User mode server. The server creates the (A) LPC port and listens and processes client requests.
• Custom DLL client. In DLL_INIT_ROUTINE, you can connect to the port configured by the server. This DLL will introduce your subsystem API, and some functions will require communication with the server.
• Support driver in kernel mode (you may not need this).

You want to save the state of a process or thread on your server or driver. If you are storing it on a server, you might need something like NtRegisterThreadTerminatePort to make sure you clear when the process or thread is complete. If you are using a driver, you need PsSetCreateProcessNotifyRoutine .

And finally, if you are on XP and below, you can add new system calls. You can do this by calling KeAddSystemServiceTable . To call system calls from user mode, you need to create such stubs (for x86):

 ; XyzCreateFooBar(__out PHANDLE FooBarHandle, __in ACCESS_MASK DesiredAccess, ...) mov eax, SYSTEM_CALL_NUMBER mov edx, 0x7ffe0300 call [edx] retn 4 

In Vista and above, you can no longer add new system service tables, because there is only room for two: kernel system calls and win32k system calls.

After a bit of googling, I found this: http://winntposix.sourceforge.net/ . I think it is very similar to what you are looking for and uses a lot of the things that I talked about.

Answer 2

I'm obsessed with my own API too. :)

And I'm glad to say that it’s nowhere as dangerous or as undocumented as some people do.:]

There is no source code for "Hello, world", because the native API does not interact with the console so easily, since it is part of the Win32 subsystem and requires client / server interaction with ports. If you need to write a console application, you need to directly contact CSRSS, whose message formats are not documented (although some of its formats can be found in ReactOS Source - this will bring you many benefits if you get acquainted with ReactOS).

I will soon send you an example that you might find interesting; for now, remember that your only option is ever to be associated with NTDLL.dll, and for this you need a driver development kit (since you need a lib file).

---

Update : check it out!

(I have a feeling that no one else will publish something completely as rebellious as this. Showing the GUI using its own API ?! I have to be crazy!)

 #include <Windows.h> typedef DWORD NTSTATUS; //These are from ReactOS typedef enum _HARDERROR_RESPONSE_OPTION { OptionAbortRetryIgnore, OptionOk, OptionOkCancel, OptionRetryCancel, OptionYesNo, OptionYesNoCancel, OptionShutdownSystem } HARDERROR_RESPONSE_OPTION, *PHARDERROR_RESPONSE_OPTION; typedef enum _HARDERROR_RESPONSE { ResponseReturnToCaller, ResponseNotHandled, ResponseAbort, ResponseCancel, ResponseIgnore, ResponseNo, ResponseOk, ResponseRetry, ResponseYes, ResponseTryAgain, ResponseContinue } HARDERROR_RESPONSE, *PHARDERROR_RESPONSE; typedef struct _UNICODE_STRING { USHORT Length; USHORT MaximumLength; PWSTR Buffer; } UNICODE_STRING, *PUNICODE_STRING; //You'll need to link to NTDLL.lib //which you can get from the Windows 2003 DDK or any later WDK NTSYSAPI VOID NTAPI RtlInitUnicodeString(IN OUT PUNICODE_STRING DestinationString, IN PCWSTR SourceString); NTSYSAPI NTSTATUS NTAPI NtRaiseHardError(IN NTSTATUS ErrorStatus, IN ULONG NumberOfParameters, IN ULONG UnicodeStringParameterMask, IN PULONG_PTR Parameters, IN HARDERROR_RESPONSE_OPTION ValidResponseOptions, OUT PHARDERROR_RESPONSE Response); #define STATUS_SERVICE_NOTIFICATION_2 0x50000018 int main() { HARDERROR_RESPONSE response; ULONG_PTR items[4] = {0}; UNICODE_STRING text, title; RtlInitUnicodeString(&text, L"Hello, NT!\r\nDo you like this?\r\n" L"This is just about as pretty as the GUI will get.\r\n" L"This message will self-destruct in 5 seconds..."); RtlInitUnicodeString(&title, L"Native Message Box!"); items[0] = (ULONG_PTR)&text; items[1] = (ULONG_PTR)&title; items[2] = (ULONG_PTR)OptionYesNo; items[3] = (ULONG_PTR)5000; NtRaiseHardError(STATUS_SERVICE_NOTIFICATION_2, ARRAYSIZE(items), 0x1 | 0x2 /*First two parameters are UNICODE_STRINGs*/, items, OptionOk /*This is ignored, since we have a custom message box.*/, &response); return 0; } 

If you have any questions, feel free to ask! I'm not afraid of the native API! :)

---

Edit 2:

If you are trying to create your own Kernel32 DLL version and download it, like Kernel32, with each process (hence the new subsystem), I just wanted to tell you that I do not think this is possible. This is quite similar to this question I asked a couple of days ago, and it seems that you cannot extend the NT PE bootloader to learn about the new subsystems, so I don’t think it will be possible.