Atomic codes
June 16th, 2007I had some fun today trying to figure out why Banner likes to hang around with .NET so much so it wouldn’t even leave. I found out that while being destroyed, something tries to send messages to the main dialog. But the main dialog is busy with destroying the banner. I added exactly two iterations of the famous win32 message loop and everything started working. I still don’t know why those messages are sent or why it’s so important they’ll be answered before the banner is destroyed or even why it happens just with the .NET installer. And don’t even ask about different synchronization methods that make it tick. So far, I’ve found only smoke signals and the fire extinguisher won’t last much longer.
Of all the signals, I liked the message loop the most. It actually points to something I’ve done wrong. I’ve starved the main dialog’s thread while creating a modeless dialog as its child. That’s why I dug in further into those two iterations of the loop and those two messages that it processes. It turns out both of them had the same identifier - 0xc0c3. Now that’s no regular WM_ message… That’s a message registered with RegisterWindowMessage. But which message is it? That’s where the fun starts. There’s no GetRegisteredWindowMessage API available and nothing on the topic comes out on Google.
So with no leads to follow I started digging. Normally, to give a certain string a specific value in Windows, an atom is created. And indeed, 0xc0c3 is in the range of named atoms. To make things even simpler, in WINE, RegisterWindowMessage simply calls GlobalAddAtom, casts ATOM to UINT and returns. Great, then GetAtomName or GlobalGetAtomName should do the trick. Only reality isn’t as bright as WINE would like us to think. It turns out RegisterWindowMessage uses a different atom table for its messages. But which atom table and how can you even specify a table with GetAtomName?
To specify a table, a low-level access to RtlLookupAtomInAtomTable is required. But that function is deep inside ntoskrnl.exe. So, up one level and you get NtUserGetAtomName which uses the same atom table as NtUserAddAtom which is the function RegisterWindowMessage calls. But that’s inside win32k.sys… Luckily, user32.dll already handles that. It has a stub that calls NtUserGetAtomName at 0×7E41FA8E. Some playing around with the second parameter which turns out to be UNICODE_STRING and the atomic table is in hands’ reach.
Engines off, coding fingers down, digging complete and the message name is MSUIM.Msg.Private. That too gets little to none results on Google, but who cares… Debugging is fun 
For any of you who’d ever want to convert a registered message into a readable name, here’s the NSIS code. Replace 0xc0c3 with the message identifier and 0×7E41FA8E with user32!NtUserGetAtomName and you’re good to go.
# the atom StrCpy $2 0xc0c3 ;System::Call user32::RegisterWindowMessage(t'test_message')i.r2 # create UNICODE_STRING System::Alloc 1008 Pop $R0 StrCpy $R1 0 StrCpy $R2 1000 IntOp $R3 $R0 + 8 System::Call *$R0(&i2R1,&i2R2,iR3) # call NtUserGetAtomName System::Call ::0x7E41FA8E(ir2,iR0)i.r1?e # parse UNICODE_STRING System::Call *$R0(&i2.r4,&i2.r3,w.r0) # print details DetailPrint "user atom's name is $0" DetailPrint "length is $4 (???)" DetailPrint "NtUserGetAtomName returned $1" Pop $1 DetailPrint "GetLastError() = $1" # done System::Free $R0
