-----Original Message----- From: Myria [mailto:myriachan@cox.net] Sent: Tuesday, August 29, 2006 10:14 AM To: 'ReactOS Development List' Subject: Re: [ros-dev] (Free)DOS subsystem
I've wanted to write an NTVDM for ReactOS, but I'm not sure when I'd ever have time.
Windows NT's DOS subsystem, NTVDM, is a user-mode program that runs on top of Win32. It runs DOS programs as Windows processes, using v86 mode to run the programs. DPMI programs are supported by asking the kernel to allocate LDT segments and running directly. (If you write a DOS32 program and know the correct addresses, you can call MessageBoxW in user32.dll and it will work.)
This differs significantly from DOSBox, which is effectively a machine emulator rather than an API translator. DOSEmu, in comparison, is essentially the same design as NTVDM but for Linux.
FreeDOS in DOSBox works very well, because the FreeDOS kernel is its normal self and is unaware that it's inside a VM. However, this doesn't work for NTVDM/DOSEmu. In these, the FreeDOS kernel will need to be heavily modified, particularly because the NT kernel handles file I/O. Programs inside the virtual DOS environment use illegal opcodes to talk to NTVDM.
FreeDOS works just fine in DOSemu. Actually that is how it got debugged by the kernel maintainer who was also the DOSemu maintainer. But no code of FreeDOS is actually special for DOSemu.
The invalid opcode handler is probably similar to int e6 in DOSemu. It is an extra API by which DOS software can use services of DOSemu.
What probably is needed is an implementation of the int 21 interface, and maybe others, in the V86 monitor.
For what I understood from reading a litle on the web is that it more or less works like this:
You have v86 code in the kernel. This can be mapped using memory paging to point to a certain address, where a DOS program is loaded.
When an interupt occurs, the V86 mode is exited (the only way a V86 task can be exited) and the V86 monitor, in this case ntvdm takes over. This looks at the state of the interrupt and acts upon this. For example when calling an int 16 function it may return the status of a key. After the interrupt has been serviced the registers are loaded with the right values, or memory is filled correctly, or something like that and the v86 task is allowed to run until the next interrupt. Of course the software APIs can also be serviced by the V86 monitor and this is probably what is done in windows.
ntvdm is special in that, unlike DOSemu, it serves as the V86 monitor for all the V86 tasks running in the session.
Imre
Melissa
----- Original Message ----- From: "James Tabor" jimtabor@adsl-64-217-116-74.dsl.hstntx.swbell.net To: "ReactOS Development List" ros-dev@reactos.org Sent: Monday, August 28, 2006 14:19 Subject: Re: [ros-dev] (Free)DOS subsystem
Imre Leber wrote:
Hi,
I guess i am new to this list.
I have been a maintainer on the FreeDOS project for more then 7 years.
I have also noticed that you are still looking into building a DOS subsystem in reactos.
After the release of FreeDOS version 1, I would like to see wether I could not take up the task to having it running under reactos.
My main idea involves using a port of DOSemu.
Is there any documentation regarding the running of a DOS subsystem in windows that I should study beforehand?
Imre
Hi Imre! Welcome to ReactOS! 8^D
The idea is still open.
We've tested Qemu under ReactOS running FreeDOS at one time and it does work. Another project that was tested, DOSBox, it works like DOSemu, http://dosbox.sourceforge.net/
Running a dos-subsystem and docs? Maybe one of the other developers can help here. You can look at the OS2 subsystem at http://svn.reactos.ru/svn/reactos/trunk/os2/ , it's incomplete, this would be your best starting point. There is allot more to it. You need to dive into the code and start reading. I do not recall any docs or books dealing with rolling your own subsystem. The best book you can get for understanding Windows or ReactOS is, Windows Internals. Understanding the Csrss subsystem would help with the os2 one.
Anyway, some projects that could be good examples on how to do this, Posix subsystem, http://sourceforge.net/projects/winntposix ,and running linux bins in windows, http://sourceforge.net/projects/keow . One of our developers is currently working with CoLinux, http://www.colinux.org/ .
Sorry I couldn't help you more here.
Thanks, James
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Not that I'm trying to rain on anyones parade, but it would be really cool if for each program you could choose to run it in either an emulation environment or in a V86 mode (ala WinNT NTVDM.EXE,) that way older DOS games could be used.
Imre Leber wrote:
-----Original Message----- From: Myria [mailto:myriachan@cox.net] Sent: Tuesday, August 29, 2006 10:14 AM To: 'ReactOS Development List' Subject: Re: [ros-dev] (Free)DOS subsystem
I've wanted to write an NTVDM for ReactOS, but I'm not sure when I'd ever have time.
Windows NT's DOS subsystem, NTVDM, is a user-mode program that runs on top of Win32. It runs DOS programs as Windows processes, using v86 mode to run the programs. DPMI programs are supported by asking the kernel to allocate LDT segments and running directly. (If you write a DOS32 program and know the correct addresses, you can call MessageBoxW in user32.dll and it will work.)
This differs significantly from DOSBox, which is effectively a machine emulator rather than an API translator. DOSEmu, in comparison, is essentially the same design as NTVDM but for Linux.
FreeDOS in DOSBox works very well, because the FreeDOS kernel is its normal self and is unaware that it's inside a VM. However, this doesn't work for NTVDM/DOSEmu. In these, the FreeDOS kernel will need to be heavily modified, particularly because the NT kernel handles file I/O. Programs inside the virtual DOS environment use illegal opcodes to talk to NTVDM.
FreeDOS works just fine in DOSemu. Actually that is how it got debugged by the kernel maintainer who was also the DOSemu maintainer. But no code of FreeDOS is actually special for DOSemu.
The invalid opcode handler is probably similar to int e6 in DOSemu. It is an extra API by which DOS software can use services of DOSemu.
What probably is needed is an implementation of the int 21 interface, and maybe others, in the V86 monitor.
For what I understood from reading a litle on the web is that it more or less works like this:
You have v86 code in the kernel. This can be mapped using memory paging to point to a certain address, where a DOS program is loaded.
When an interupt occurs, the V86 mode is exited (the only way a V86 task can be exited) and the V86 monitor, in this case ntvdm takes over. This looks at the state of the interrupt and acts upon this. For example when calling an int 16 function it may return the status of a key. After the interrupt has been serviced the registers are loaded with the right values, or memory is filled correctly, or something like that and the v86 task is allowed to run until the next interrupt. Of course the software APIs can also be serviced by the V86 monitor and this is probably what is done in windows.
ntvdm is special in that, unlike DOSemu, it serves as the V86 monitor for all the V86 tasks running in the session.
Imre
Melissa
----- Original Message ----- From: "James Tabor" jimtabor@adsl-64-217-116-74.dsl.hstntx.swbell.net To: "ReactOS Development List" ros-dev@reactos.org Sent: Monday, August 28, 2006 14:19 Subject: Re: [ros-dev] (Free)DOS subsystem
Imre Leber wrote:
Hi,
I guess i am new to this list.
I have been a maintainer on the FreeDOS project for more then 7 years.
I have also noticed that you are still looking into building a DOS subsystem in reactos.
After the release of FreeDOS version 1, I would like to see wether I could not take up the task to having it running under reactos.
My main idea involves using a port of DOSemu.
Is there any documentation regarding the running of a DOS subsystem in windows that I should study beforehand?
Imre
Hi Imre! Welcome to ReactOS! 8^D
The idea is still open.
We've tested Qemu under ReactOS running FreeDOS at one time and it does work. Another project that was tested, DOSBox, it works like DOSemu, http://dosbox.sourceforge.net/
Running a dos-subsystem and docs? Maybe one of the other developers can help here. You can look at the OS2 subsystem at http://svn.reactos.ru/svn/reactos/trunk/os2/ , it's incomplete, this would be your best starting point. There is allot more to it. You need to dive into the code and start reading. I do not recall any docs or books dealing with rolling your own subsystem. The best book you can get for understanding Windows or ReactOS is, Windows Internals. Understanding the Csrss subsystem would help with the os2 one.
Anyway, some projects that could be good examples on how to do this, Posix subsystem, http://sourceforge.net/projects/winntposix ,and running linux bins in windows, http://sourceforge.net/projects/keow . One of our developers is currently working with CoLinux, http://www.colinux.org/ .
Sorry I couldn't help you more here.
Thanks, James
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Hmm, I'm not sure if it is really even necessary to directly support DOS programs. One thing I thought of that I should mention here is that Microsoft doesn't even support DOS programs at all with their latest Windows versions (Windows XP and Server 2003 x64 Editions). However, that isn't really a problem since there are alternatives to Windows' DOS support (which wasn't really even that great in some ways in Windows 2000 or XP to begin with anyway).
I don't know why, but for some reason I'm thinking it would just kind of be a waste of time to implement NTVDM for ReactOS if the developers working on it would be able to work on other parts of ReactOS. Hmm, it might be at least somewhat worthwhile to work on a NTVDM alternate that could run on both ReactOS and Windows (possibly even x64 editions), though.
Well, just my opinions on the matter.
-ShadowFlare
On 8/29/06, Nate DeSimone desimn@rpi.edu wrote:
Not that I'm trying to rain on anyones parade, but it would be really cool if for each program you could choose to run it in either an emulation environment or in a V86 mode (ala WinNT NTVDM.EXE,) that way older DOS games could be used.
Imre Leber wrote:
-----Original Message----- From: Myria [mailto:myriachan@cox.net] Sent: Tuesday, August 29, 2006 10:14 AM To: 'ReactOS Development List' Subject: Re: [ros-dev] (Free)DOS subsystem
I've wanted to write an NTVDM for ReactOS, but I'm not sure when I'd
ever
have time.
Windows NT's DOS subsystem, NTVDM, is a user-mode program that runs on
top
of Win32. It runs DOS programs as Windows processes, using v86 mode to
run
the programs. DPMI programs are supported by asking the kernel to
allocate
LDT segments and running directly. (If you write a DOS32 program and
know
the correct addresses, you can call MessageBoxW in user32.dll and it
will
work.)
This differs significantly from DOSBox, which is effectively a machine emulator rather than an API translator. DOSEmu, in comparison, is essentially the same design as NTVDM but for Linux.
FreeDOS in DOSBox works very well, because the FreeDOS kernel is its
normal
self and is unaware that it's inside a VM. However, this doesn't work
for
NTVDM/DOSEmu. In these, the FreeDOS kernel will need to be heavily modified, particularly because the NT kernel handles file
I/O. Programs
inside the virtual DOS environment use illegal opcodes to talk to
NTVDM.
FreeDOS works just fine in DOSemu. Actually that is how it got debugged
by the kernel maintainer who was also the DOSemu maintainer. But no code of FreeDOS is actually special for DOSemu.
The invalid opcode handler is probably similar to int e6 in DOSemu. It
is an extra API by which DOS software can use services of DOSemu.
What probably is needed is an implementation of the int 21 interface,
and maybe others, in the V86 monitor.
For what I understood from reading a litle on the web is that it more or
less works like this:
You have v86 code in the kernel. This can be mapped using memory paging
to point to a certain address, where a DOS program is loaded.
When an interupt occurs, the V86 mode is exited (the only way a V86 task
can be exited) and the V86 monitor, in this case ntvdm takes over. This looks at the state of the interrupt and acts upon this. For example when calling an int 16 function it may return the status of a key. After the interrupt has been serviced the registers are loaded with the right values, or memory is filled correctly, or something like that and the v86 task is allowed to run until the next interrupt. Of course the software APIs can also be serviced by the V86 monitor and this is probably what is done in windows.
ntvdm is special in that, unlike DOSemu, it serves as the V86 monitor
for all the V86 tasks running in the session.
Imre
Melissa
----- Original Message ----- From: "James Tabor" jimtabor@adsl-64-217-116-74.dsl.hstntx.swbell.net To: "ReactOS Development List" ros-dev@reactos.org Sent: Monday, August 28, 2006 14:19 Subject: Re: [ros-dev] (Free)DOS subsystem
Imre Leber wrote:
Hi,
I guess i am new to this list.
I have been a maintainer on the FreeDOS project for more then 7
years.
I have also noticed that you are still looking into building a DOS subsystem in reactos.
After the release of FreeDOS version 1, I would like to see wether I could not take up the task to having it running under reactos.
My main idea involves using a port of DOSemu.
Is there any documentation regarding the running of a DOS subsystem
in
windows that I should study beforehand?
Imre
Hi Imre! Welcome to ReactOS! 8^D
The idea is still open.
We've tested Qemu under ReactOS running FreeDOS at one time and it
does
work. Another project that was tested, DOSBox, it works like DOSemu,
http://dosbox.sourceforge.net/
Running a dos-subsystem and docs? Maybe one of the other developers
can
help here. You can look at the OS2 subsystem at http://svn.reactos.ru/svn/reactos/trunk/os2/ ,
it's
incomplete, this would be your best starting point. There is allot more to it. You need to dive
into
the code and start reading. I do not recall any docs or books dealing with rolling your
own
subsystem. The best book you can get for understanding Windows or ReactOS is, Windows
Internals.
Understanding the Csrss subsystem would help with the os2 one.
Anyway, some projects that could be good examples on how to do this,
Posix
subsystem, http://sourceforge.net/projects/winntposix ,and running linux bins in windows, http://sourceforge.net/projects/keow . One of our developers is
currently
working with CoLinux, http://www.colinux.org/ .
Sorry I couldn't help you more here.
Thanks, James
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
On Tue, 2006-08-29 at 19:34 -0600, ShadowFlare wrote:
Hmm, I'm not sure if it is really even necessary to directly support DOS programs. One thing I thought of that I should mention here is that Microsoft doesn't even support DOS programs at all with their latest Windows versions (Windows XP and Server 2003 x64 Editions). However, that isn't really a problem since there are alternatives to Windows' DOS support (which wasn't really even that great in some ways in Windows 2000 or XP to begin with anyway).
I don't know why, but for some reason I'm thinking it would just kind of be a waste of time to implement NTVDM for ReactOS if the developers working on it would be able to work on other parts of ReactOS. Hmm, it might be at least somewhat worthwhile to work on a NTVDM alternate that could run on both ReactOS and Windows (possibly even x64 editions), though.
Well, just my opinions on the matter.
-ShadowFlare
Doesn’t NTVDM also handle WOW (Windows on Windows) sessions with 16-bit Windows applications? I seem to recall when I worked at AT&T having (quite a few) applications that the only way they could be killed was to kill NTVDM and another process (old[something]app or something?) and then restart the application. In any case, there are still quite a bit of software applications that are available online that would make use of these types of subsystems—and it would be nice, I think, to continue supporting something that Microsoft does not, and improve upon it, because that would just be another point in favor of getting people to use free software. Right? :-)
— Mike
-- The fd0man™—The Magical Floppy Man! (fd0man@gmail.com) "One world, one web, one program" —Microsoft promotional ad "Ein Volk, ein Reich, ein Fuehrer" —Adolf Hitler
On 8/29/06, fd0man™—The Magical Floppy Man fd0man@gmail.com wrote:
On Tue, 2006-08-29 at 19:34 -0600, ShadowFlare wrote:
Hmm, I'm not sure if it is really even necessary to directly support DOS programs. One thing I thought of that I should mention here is that Microsoft doesn't even support DOS programs at all with their latest Windows versions (Windows XP and Server 2003 x64 Editions). However, that isn't really a problem since there are alternatives to Windows' DOS support (which wasn't really even that great in some ways in Windows 2000 or XP to begin with anyway).
I don't know why, but for some reason I'm thinking it would just kind of be a waste of time to implement NTVDM for ReactOS if the developers working on it would be able to work on other parts of ReactOS. Hmm, it might be at least somewhat worthwhile to work on a NTVDM alternate that could run on both ReactOS and Windows (possibly even x64 editions), though.
Well, just my opinions on the matter.
-ShadowFlare
Doesn't NTVDM also handle WOW (Windows on Windows) sessions with 16-bit Windows applications? I seem to recall when I worked at AT&T having (quite a few) applications that the only way they could be killed was to kill NTVDM and another process (old[something]app or something?) and then restart the application. In any case, there are still quite a bit of software applications that are available online that would make use of these types of subsystems—and it would be nice, I think, to continue supporting something that Microsoft does not, and improve upon it, because that would just be another point in favor of getting people to use free software. Right? :-)
— Mike
Heh, I still think that an actual NTVDM implementation might not necessarily be the best solution, though. Well, if it could run on x64 editions of Windows then it would be fine, but if it couldn't, then that's why I was suggesting an alternative. Of course, the alternative still wouldn't have to be just for DOS programs either, it could also be for 16-bit Windows applications as well.
Hmm, if there could be an implementation with no kernel-mode code or at least something that could somehow work when running AMD64 CPUs in a 64-bit OS, then that would be a good thing to work toward. Then if ReactOS was later made to support the API for Windows x64, then it could have 16-bit app support, while Microsoft's 64-bit OS doesn't. :)
-ShadowFlare
On Tue, 2006-08-29 at 20:10 -0600, ShadowFlare wrote:
Heh, I still think that an actual NTVDM implementation might not necessarily be the best solution, though. Well, if it could run on x64 editions of Windows then it would be fine, but if it couldn't, then that's why I was suggesting an alternative. Of course, the alternative still wouldn't have to be just for DOS programs either, it could also be for 16-bit Windows applications as well.
Hmm, if there could be an implementation with no kernel-mode code or at least something that could somehow work when running AMD64 CPUs in a 64-bit OS, then that would be a good thing to work toward. Then if ReactOS was later made to support the API for Windows x64, then it could have 16-bit app support, while Microsoft's 64-bit OS doesn't. :)
Well, as I understand it, in long mode the v86 Mode functionality of 32-bit protected mode systems is no longer available, though that does not, I do not think, preclude creating a new NTVDM type system that is compatible with the 32-bit current NTVDM. The only real difference is that you would more likely than not have to have the v86 processes running in an actual virtual machine with a monitor, though it should (in theory) be possible to do that and retain compatibility, possibly at a slight loss of speed.
Of course, there are a whole slew of choices that are available then; should it be a virtual machine, or should it perhaps have some sort of dynamic recompliation mechanism that would "turn" it into a native executable, perhaps even ahead of time? IANAP (at least, not a good one, yet) and so I am not sure what would really be the more feasible option, and other decisions would need to be made after that. But, yes, it would be nice to have a "one-up" of sorts on Microsoft.
— Mike
-- The fd0man™—The Magical Floppy Man! (fd0man@gmail.com) "One world, one web, one program" —Microsoft promotional ad "Ein Volk, ein Reich, ein Fuehrer" —Adolf Hitler
Hmm, I'm not sure if it is really even necessary to directly support DOS programs. One thing I thought of that I should mention here is that Microsoft doesn't even support DOS programs at all with their latest Windows versions (Windows XP and Server 2003 x64 Editions). However, that isn't really a problem since there are alternatives to Windows' DOS support (which wasn't really even that great in some ways in Windows 2000 or XP to begin with anyway).
The 2 main reasons to bother with them are to support old games and install programs. DOS application programs like WordPerfect 5.1 have long since been replaced. Games, however, never really become obsolete.
I think a large part of why Windows didn't run DOS programs too well was that they didn't implement what games needed. Games need Sound Blaster, VESA, and joysticks. These can be done, but they didn't bother. I'm not blaming Microsoft - I doubt that spending time on it would've helped their bottom line.
I don't know why, but for some reason I'm thinking it would just kind of be a waste of time to implement NTVDM for ReactOS if the developers working on it would be able to work on other parts of ReactOS. Hmm, it might be at least somewhat worthwhile to work on a NTVDM alternate that could run on both ReactOS and Windows (possibly even x64 editions), though.
NTVDM is basically a "marketing" project. If eventually ReactOS runs old DOS games better than NT, then that becomes a "selling" point. Of course, that's a long way off...
NTVDM for Win64 is basically impossible for exactly one reason: Win64 x86-64 does not implement the LDT. It does "xor eax, eax \ lldt eax" during startup and that's the end of it. NtSetLdtEntries is a stub, the trap frame doesn't have an LDT field, and the context-switching code does not have code to switch LDT values.
If you tried to implement it by patching the kernel, you'd run into PatchGuard. If you break PatchGuard, your code would only work until the next second Tuesday.
Finally, even if you got around PatchGuard, Vista64 requires drivers to be signed, and such a system is antithetical to open-source. People wouldn't be able to compile and run the code themselves (unless they used test signing, which would disable playback of HD-DVDs).
The issue that V86 mode doesn't work in Long Mode isn't a big deal. Modern machines can run the programs that ran in real mode in an emulator faster than they ran on their original hardware. DOS32 games require more performance, but you can run those directly.
Well, just my opinions on the matter.
-ShadowFlare
Melissa
http://sourceforge.net/projects/ntvdm Requirement with Xp to install this stuff to get the NTVDM functioning anywhere near good.
NTVDM was never fully functional as it could have been from the starting line.
At least some of out NTVDM is aquireable from other opensource projects.
Peter Dolding Myria wrote:
Hmm, I'm not sure if it is really even necessary to directly support DOS programs. One thing I thought of that I should mention here is that Microsoft doesn't even support DOS programs at all with their latest Windows versions (Windows XP and Server 2003 x64 Editions). However, that isn't really a problem since there are alternatives to Windows' DOS support (which wasn't really even that great in some ways in Windows 2000 or XP to begin with anyway).
The 2 main reasons to bother with them are to support old games and install programs. DOS application programs like WordPerfect 5.1 have long since been replaced. Games, however, never really become obsolete.
I think a large part of why Windows didn't run DOS programs too well was that they didn't implement what games needed. Games need Sound Blaster, VESA, and joysticks. These can be done, but they didn't bother. I'm not blaming Microsoft - I doubt that spending time on it would've helped their bottom line.
I don't know why, but for some reason I'm thinking it would just kind of be a waste of time to implement NTVDM for ReactOS if the developers working on it would be able to work on other parts of ReactOS. Hmm, it might be at least somewhat worthwhile to work on a NTVDM alternate that could run on both ReactOS and Windows (possibly even x64 editions), though.
NTVDM is basically a "marketing" project. If eventually ReactOS runs old DOS games better than NT, then that becomes a "selling" point. Of course, that's a long way off...
NTVDM for Win64 is basically impossible for exactly one reason: Win64 x86-64 does not implement the LDT. It does "xor eax, eax \ lldt eax" during startup and that's the end of it. NtSetLdtEntries is a stub, the trap frame doesn't have an LDT field, and the context-switching code does not have code to switch LDT values.
If you tried to implement it by patching the kernel, you'd run into PatchGuard. If you break PatchGuard, your code would only work until the next second Tuesday.
Finally, even if you got around PatchGuard, Vista64 requires drivers to be signed, and such a system is antithetical to open-source. People wouldn't be able to compile and run the code themselves (unless they used test signing, which would disable playback of HD-DVDs).
The issue that V86 mode doesn't work in Long Mode isn't a big deal. Modern machines can run the programs that ran in real mode in an emulator faster than they ran on their original hardware. DOS32 games require more performance, but you can run those directly.
Well, just my opinions on the matter.
-ShadowFlare
Melissa
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev