Re: [ros-dev] ReactOS versioning

[Hermès BÉLUSCA - MAÏTO]

I fully second Alex.

 

Concerning point 1 of Timo’s ideas, I was also thinking about making some kind of wizard
in which you say which exe you want to run, and choose in some lists the compatibility
features.

 

Cheers,

Hermès.

 

De : Ros-dev [mailto:[e-mail]] De la part de Alex Ionescu
Envoyé : samedi 7 mars 2015 15:07
À : ReactOS Development List
Objet : Re: [ros-dev] ReactOS versioning

 

I agree 100%. A combination of the Shim Engine, MinWin/API Sets and Fusion/SxS would help
us achieve these goals while at the same time implementing key missing Windows features we
don't yet have.




Best regards,
Alex Ionescu

 

On Sat, Mar 7, 2015 at 4:44 AM, Timo Kreuzer <timo.kreuzer(a)web.de> wrote:

Hi,

Here's the promised suggestion regarding how we handle versioning problems in reactos.
It has some relationship to the tree restructure.

Since some time we now run into issues with our targeted Windows version. This is both
wine dlls, as well as applications that refuse to run due to reactos being limited to
Windows server 2003 SP2.

I think many of us, me included, see more in ReactOS than an academic research project, or
a nice way for 3rd party companies to cheaply get insight into how the Windows kernel
works. So we are interested in making it an actually useful operating system. To achieve
this goal, it is obviously important to make it run modern Windows applications.

The current approach of pure Windows 2003 Server SP2 compatibility on user mode side is a
dead end. Our target OS version is starting to become a fossil. With time more and more
applications will simply refuse to work on it. Even wine DLLs start to require Vista APIs
and their number will most likely increase.

So what can we do? It is obvious, that we cannot instantaniously switch all user mode to
Windows 7/8/10 compatibility, due to the amount of required work, especially regarding
missing kernel features.

The wine approach is just adding whatever is needed, creating a Windows version chimera.
It has already been discussed here and shown to be a problem, since it can easily fool
applications into believing they run on Vista or Windows 7, making them demand all the
modern features, which we cannot provide, thus failing to run, while they would run
flawlessly, when being provided a pure Windows 2003 environment, restricting itself to
this functionality. So this is also not a very good way, either.

So the conclusion is, that we need a mechanism, that allows us to control this, providing
individual applications with what they require, while leaving others in a more restricted
environment. And at the same time allowing our internal/wine DLLs to make use of higher
version functionality.

Suggested approach:

1. We need a method to specify which application should be run in which environment. We
should probably use the same mechanism that is used on Windows. Compatibility information
is stored in a registry key HKCU\Software\Microsodt\Windows
NT\CurrentVersion\AppCompatFlags\... The trick is to make this easy / transparent for the
user. A right-click -> properties -> compatibility approach should for now probably
be the easiest thing, even if it requires the user to actively make this setting. A larger
app compatibility database would be nice, but it would be difficult to figure out what
application is being run. And it's also a problem to maintain such a list. Potential
solutions: detect failures to load due to missing imports and app crashes and invoke a
"compatibility assistent" in that case. Detect first-run of a new application
and try to identify it, either based on a hash or based on PE version information.

2. We need a way to provide the application transparently with the environment we want to
give it. In terms of DLL exports this could be done on the loader side, making it chose
the right DLL, potentially adding a suffix to the DLL name or selecting a different folder
other than system32. While this will most likely work good in the majority of cases, it is
not 100% transparent. Therefore a mechanism in the kernel, using file system redirection,
like it already exists on 64 bit Windows for WoW64, seems to be a more promising approach.
The file system redirection would redirect system32 into merged folders, containing the
version specific DLLs, while everything that is not existing in this folder will be taken
from the original system32. Potential naming scheme: system32.601 system32.602, etc.

3. We need a method to create and maintain the required DLLs for different OS versions.
Preferably avoiding bloat by sharing common code in common "parent" DLLs. But
also allowing to still plug the DLLs into the related Windows version for testing. This
can be tricky. I suggest a DLL import forwarding scheme. This is both to avoid bloat, i.e.
avoid to compile and deploy all full blown DLLs for all OS versions, as well as creating a
better organized system. So each DLL, lets say ADVAPI32, as located in different version
specific system32 folders, would mainly/only consist of forwarders to a "parent"
DLL. On Windows we can see this being developed similarly, using "api sets" and
redirections made by the loader. Cloning this mechanism 1:1 might not be the right thing
though, since it does not address all our requirements. So instead I suggest proving our
own custom "parent DLLs". While these could be organized the same way as on
Windows 2003, this is probably not optimal. Instead I suggest merging stuff together into
1 or few DLLs (similar to how stuff was combined in kernelbase.dll)
This might looks like this (note, that the names are just quickly made up names, I
don't claim that they are good)
- user32/gdi32 -> ros-win32-core.dll
- kernel32/advapi32 -> ros-kernel-base.dll
- msvcr* -> ros-crt.dll
- ntdll -> ros-ntdll (the kernel would need to load this one)

This also allows our DLLs to make use of higher version APIs, by linking them to the
parent DLL.
Now this obviously introduces a problem when trying to run individual DLLs within a
Windows system. To still be able to do this for testing purposes, we need to make sure
that they still run there.
First, if they import from a custom ros-* DLL, it won't run on Windows without that
DLL. So we need the possibility to either statically link these functions, or compile a
"helper DLL", that contains these functions, so our DLLs can be run on Windows.
If we used a common parent DLL, this also creates the problem of DLL initialization. e.g.
the DLL parent for kernel32 and advapi32 would do the initialization for both of these, so
this would not be suitable to use when replacing only one of the DLLs. Instead DLL
initialization could be done by calling a specific initialization function in the parent
DLL from the child DLL. So kernel32!DllMain would call ros-kernel-base!DllMain_kernel32,
passing the original parameters as well as a version number, that the parent DLL can use
to do version specific initialization. This way The parent DLL would only do the kernel32
initialization, when the related kernel32 child DLL was used, the advapi32 initialization
would not be done.
There is still a problem: relocation. So we would need to make sure we chose base
addresses that still allow us to plug the stuff into Windows without causing everything to
relocate, which often simply doesn't work.
As an alternative, we should provide a compile time switch to compile specific DLLs in a
self-contained way.

In terms of structure we could use the MS api-sets as a base for static libraries. Then we
can link these either into the parent DLLs like ros-kernel-base.dll or - when a compile
switch is given - to link together fully self contained DLLs.

I am really not interested in answers like "This is not what WIndows does!",
"THIS CANNOT WORK!!!", "You are a ***** even suggesting this",
"What if application x parses the import table and disassembles the DLL to hook into
internal functions, ...":

I am only interested in *constructive* comments.
Everything else: -> /dev/null

Thanks,
Timo



_______________________________________________
Ros-dev mailing list
Ros-dev(a)reactos.org
http://www.reactos.org/mailman/listinfo/ros-dev