Let's push winetests farther!

[Jérôme Gardou]

As you all know, reactos is tested against the wine test suite. While
this is a great tool, I think this is not enough to be sure that ReactOS
is fully tested.
I've been trying to fix some kernel32 tests. As you all know, in ReactOS
it is written on top of ntdll. So I wanted to fix some tests, and then a
question arose : is it that kernel32 calls ntdll the wrong way, or is it
that ntdll does the wrong job? The only way to verify this would be to
test kernel32.dll out of reactOS, meaning replacing the windows one with
the reactos one, and run the test.
To be brief : I think that a weekly "reactos dlls against windows" test
would be very helpful to be sure that things are done the right way!
I began with the kernel32:file winetest. I managed to not make it crash
when used on winXP, the resulting patch is there :
http://www.reactos.org/bugzilla/show_bug.cgi?id=4728 and the result of
this test is attached there. As miracles happen, it also fixes some
tests on ReactOS. Seems like this is a good approach.
Jérôme Gardou (aka zefklop on IRC and forums)

[Timo Kreuzer]

Hi,
Some dlls cannot be properly tested on Window. These are:
ntdll: it uses direct systemcalls and the system call numbers would need
to match the windows system you test it on. If you are very lucky you
could find one version of win2k3 that matches our syscalls, but I
wouldn't bet on that.
kernel32: it's communication with csrss is not Windows compatible. We
would have to fix (rewrite) our csrss, something that would requite lots
of work. And noone loves csrss. (We accept patches. ;-))
gdi32: It loads and partly works on Windows XP, as the handle manager
win32k syscall numbers are now mostly XP compatible. I'm not sure
whether this is still true, as there were some changes recently and I
don't know on what Windows version those were based. But the winetest
results are a desaster, it even fails CreateDC.
user32: Doesn't work, for a lot of reasons. user32 is a major user of
kernel<->usermode shared data. Our data structures are not anywhere like
Windows ones and probably will never be 100%. Same for the handle
manager. Also incompatible is communication with csrss again and several
syscalls that are incompatible to Windows. And finally we don't
(didn't?) properly initialize the kernel callbacks, making it crash on
systemcalls.That may actually be fixed by Jim's recent commits.
Btw: what are the results you sent? kernel32 on windows XP? I wonder how
you managed to get any console output as that wouldn't work with our
kernel32 on Windows. For me it doesn't load, even with your patch.
WinDbg shows the following debug message:
"output-i386\modules\rostests\winetests\kernel32\kernel32.dll
(dll\win32\kernel32\misc\dllmain.c:203) CSR Failed to give us a console"
So I guess you made a mistake ;-)
Regards,
Timo
Jérôme Gardou wrote:
>   As you all know, reactos is tested against the wine
> test suite. While
>  this is a great tool, I think this is not enough to be sure that
>  ReactOS is fully tested.
>  I've been trying to fix some kernel32 tests. As you all know, in
>  ReactOS it is written on top of ntdll. So I wanted to fix some tests,
>  and then a question arose : is it that kernel32 calls ntdll the wrong
>  way, or is it that ntdll does the wrong job? The only way to verify
>  this would be to test kernel32.dll out of reactOS, meaning replacing
>  the windows one with the reactos one, and run the test.
>  To be brief : I think that a weekly "reactos dlls against windows"
>  test would be very helpful to be sure that things are done the right way!
>  I began with the kernel32:file winetest. I managed to not make it
>  crash when used on winXP, the resulting patch is there :
>  http://www.reactos.org/bugzilla/show_bug.cgi?id=4728 and the result of
>  this test is attached there. As miracles happen, it also fixes some
>  tests on ReactOS. Seems like this is a good approach.
>  Jérôme Gardou (aka zefklop on IRC and forums)
>  ------------------------------------------------------------------------
>  _______________________________________________
>  Ros-dev mailing list
>  Ros-dev(a)reactos.org
>  http://www.reactos.org/mailman/listinfo/ros-dev

[Jérôme Gardou]

I do get this error too, but that does not stop me from doing some
tests. What I did was to recompile kernel32 as kernel32_test and link
the winetest to it instead of kernel32, and launch it in winXP. This
sounds really dirty as it is, but gave me good result. The fixed tests
on reactos side was a mistake.
Timo Kreuzer wrote:
>   Hi,
>  Some dlls cannot be properly tested on Window. These are:
>  ntdll: it uses direct systemcalls and the system call numbers would
>  need to match the windows system you test it on. If you are very lucky
>  you could find one version of win2k3 that matches our syscalls, but I
>  wouldn't bet on that.
>  kernel32: it's communication with csrss is not Windows compatible. We
>  would have to fix (rewrite) our csrss, something that would requite
>  lots of work. And noone loves csrss. (We accept patches. ;-))
>  gdi32: It loads and partly works on Windows XP, as the handle manager
>  win32k syscall numbers are now mostly XP compatible. I'm not sure
>  whether this is still true, as there were some changes recently and I
>  don't know on what Windows version those were based. But the winetest
>  results are a desaster, it even fails CreateDC.
>  user32: Doesn't work, for a lot of reasons. user32 is a major user of
>  kernel<->usermode shared data. Our data structures are not anywhere
>  like Windows ones and probably will never be 100%. Same for the handle
>  manager. Also incompatible is communication with csrss again and
>  several syscalls that are incompatible to Windows. And finally we
>  don't (didn't?) properly initialize the kernel callbacks, making it
>  crash on systemcalls.That may actually be fixed by Jim's recent commits.
>  Btw: what are the results you sent? kernel32 on windows XP? I wonder
>  how you managed to get any console output as that wouldn't work with
>  our kernel32 on Windows. For me it doesn't load, even with your patch.
>  WinDbg shows the following debug message:
>  "output-i386\modules\rostests\winetests\kernel32\kernel32.dll
>  (dll\win32\kernel32\misc\dllmain.c:203) CSR Failed to give us a console"
>  So I guess you made a mistake ;-)
>  Regards,
>  Timo
>  Jérôme Gardou wrote:
> >   As you all know, reactos is tested against the
> > wine test suite. While
> >  this is a great tool, I think this is not enough to be sure that
> >  ReactOS is fully tested.
> >  I've been trying to fix some kernel32 tests. As you all know, in
> >  ReactOS it is written on top of ntdll. So I wanted to fix some tests,
> >  and then a question arose : is it that kernel32 calls ntdll the wrong
> >  way, or is it that ntdll does the wrong job? The only way to verify
> >  this would be to test kernel32.dll out of reactOS, meaning replacing
> >  the windows one with the reactos one, and run the test.
> >  To be brief : I think that a weekly "reactos dlls against windows"
> >  test would be very helpful to be sure that things are done the right
> >  way!
> >  I began with the kernel32:file winetest. I managed to not make it
> >  crash when used on winXP, the resulting patch is there :
> >  http://www.reactos.org/bugzilla/show_bug.cgi?id=4728 and the result
> >  of this test is attached there. As miracles happen, it also fixes
> >  some tests on ReactOS. Seems like this is a good approach.
> >  Jérôme Gardou (aka zefklop on IRC and forums)
> >  ------------------------------------------------------------------------
> >  _______________________________________________
> >  Ros-dev mailing list
> >  Ros-dev(a)reactos.org
> >  http://www.reactos.org/mailman/listinfo/ros-dev 
>  ------------------------------------------------------------------------
>  _______________________________________________
>  Ros-dev mailing list
>  Ros-dev(a)reactos.org
>  http://www.reactos.org/mailman/listinfo/ros-dev

[Jérôme Gardou]

I think I should explain more precisely what I want to do :
My goal is to make all kernel32:file winetests pass. What I plan to do
is : getting the current implementation pass this tests on winXP. I
don't really care if the stuff made to avoid crashes is not applied, but
the functional cahnges should, IMO, get into trunk. Currently, there are
120+ tests that don't pass. The goal is to get this number to 0.
As a side effect, it *might* make more tests pass in reactOS, although I
expect the number of failures to increase. Once it is done, as making
ntdll work on winXP/2003 would require too much work, I plan to use the
native winXP fat driver in reactOS, and fix ntoskrnl to get them right.
(whether reactOS will boot with it is another story, and fixes for that
would be very positive IMO)
Final step would be to fix the fastfat driver...
Regards
Jérôme Gardou (aka zefklop)