[ros-dev] Broken Message Dispatching in Win32k

[Alex Ionescu]

Hi,

After applying my user-mode callback patch, I sometimes see a bugcheck 
caused by my debugging code. It seems that we do user-mode callbacks 
with Kernel APCs disabled (just like we also did them with PreviousMode 
== KernelMode, thank God that I had to "break" HEAD to find this huge 
bug... but that's another story). The old code "worked" because it did 
not check for this serious mistake. Normally, I've noticed that messages 
are sent through co_IntSendMessage which is called with KernelApcDisable 
== -1. That function calls some unlocking functions and then does the 
Callback, which is called with KernelApcDisable == 0, which is good. 
However, co_IntSendMessage is sometimes being called from 
co_MsqPeekHardwareMessage.

   DPRINT1("ApcState: %x\n", KeGetCurrentThread()->KernelApcDisable);
   WaitObjects[1] = MessageQueue->NewMessages;
   WaitObjects[0] = &HardwareMessageQueueLock;
   do
   {
      UserLeaveCo();

      WaitStatus = KeWaitForMultipleObjects(2, WaitObjects, WaitAny, 
UserRequest,
                                            UserMode, FALSE, NULL, NULL);

      UserEnterCo();

      DPRINT1("ApcState: %x\n", KeGetCurrentThread()->KernelApcDisable);
      while (co_MsqDispatchOneSentMessage(MessageQueue))
      {
         DPRINT1("ApcState: %x\n", KeGetCurrentThread()->KernelApcDisable);
      }
      DPRINT1("ApcState: %x\n", KeGetCurrentThread()->KernelApcDisable);
   }
   while (NT_SUCCESS(WaitStatus) && STATUS_WAIT_0 != WaitStatus);

In this loop, the first KernelApcState is -1. After the wait, it becomes 
-2. Now normally there is nothing to dispatch, so the function 
continues, and the -2 later becomes a -3 after another lock, then 
gradually goes down to -2, then back to -1. So the function enters with 
-1 and exits with -1, which is normal. However, if the Message Queue 
*does* have a message on it, then co_MsqDispatchOneSentMessage ends up 
being called. Remember that after the wait we're now at -2. So that 
function will then call co_IntSendMessage at -2, which will lower ir it 
to -1 before the callback. But now the callback runs at -1, which means 
Kernel APCs are disabled... so we bugcheck.

I have absolutely NO knowledge of Win32k Message Queues/MesssageSending, 
but something defintely seems wrong to me here...can anyone help please?

Best regards,
Alex Ionescu