I was in a strange mood and decided to try to implement something Alex wanted.
I've done the leg-work on this, but need someone familiar with boot-up and particularly smp to review this patch. I do not have access to smp, and do not wish to break it.
This patch implements the /3G switch from within multiboot.S, which is necessary in order to configure the page tables correctly.
I noticed some code in _main() that was processing a 3G switch on the command-line, and I can't understand how it could possibly work because it's not readjusting the page tables, nor does it transition to the 0xC0000000 address space that I can see, so I don't think that code actually works. If it does, I'm glad to be proved wrong.
The patch I've written makes an assumption that lowmem is available when application processors execute the code. I can't seem to figure out how or where the application processor(s) actually get told to start executing, so I don't know if this assumption is true. However, if it's not true, if someone could point me to the code that inits the application processors, I can configure them to pass the needed info via %ebp or something.
Finally, I haven't actually tried booting this code. I wanted someone to review it first for obvious blunders, or to tell me, if necessary, that I've wasted my time ;) If I'm on the right track then, when I'm feeling motivated enough again, I will actually try booting it.
Royce Mitchell III wrote:
I was in a strange mood and decided to try to implement something Alex wanted.
=)
I've done the leg-work on this, but need someone familiar with boot-up and particularly smp to review this patch. I do not have access to smp, and do not wish to break it.
This patch implements the /3G switch from within multiboot.S, which is necessary in order to configure the page tables correctly.
I noticed some code in _main() that was processing a 3G switch on the command-line, and I can't understand how it could possibly work because it's not readjusting the page tables, nor does it transition to the 0xC0000000 address space that I can see, so I don't think that code actually works. If it does, I'm glad to be proved wrong.
I added it, simply to see how ntoskrnl would work with dynamic address.
The patch I've written makes an assumption that lowmem is available when application processors execute the code. I can't seem to figure out how or where the application processor(s) actually get told to start executing, so I don't know if this assumption is true. However, if it's not true, if someone could point me to the code that inits the application processors, I can configure them to pass the needed info via %ebp or something.
Finally, I haven't actually tried booting this code. I wanted someone to review it first for obvious blunders, or to tell me, if necessary, that I've wasted my time ;) If I'm on the right track then, when I'm feeling motivated enough again, I will actually try booting it.
I'll test it on qemu...can't vouch for smp though.
If it works I'll get to work on relocation when I get back home.
Best regards, Alex Ionescu
Royce Mitchell III schrieb:
I was in a strange mood and decided to try to implement something Alex wanted.
I've done the leg-work on this, but need someone familiar with boot-up and particularly smp to review this patch. I do not have access to smp, and do not wish to break it.
This patch implements the /3G switch from within multiboot.S, which is necessary in order to configure the page tables correctly.
I noticed some code in _main() that was processing a 3G switch on the command-line, and I can't understand how it could possibly work because it's not readjusting the page tables, nor does it transition to the 0xC0000000 address space that I can see, so I don't think that code actually works. If it does, I'm glad to be proved wrong.
The patch I've written makes an assumption that lowmem is available when application processors execute the code. I can't seem to figure out how or where the application processor(s) actually get told to start executing, so I don't know if this assumption is true. However, if it's not true, if someone could point me to the code that inits the application processors, I can configure them to pass the needed info via %ebp or something.
Finally, I haven't actually tried booting this code. I wanted someone to review it first for obvious blunders, or to tell me, if necessary, that I've wasted my time ;) If I'm on the right track then, when I'm feeling motivated enough again, I will actually try booting it.
Ros-dev mailing list Ros-dev@reactos.com http://reactos.com:8080/mailman/listinfo/ros-dev
Hi,
I think your patch will never work. There are used some offsets which based on the '/3GB' switch and other ones are hard coded by KERNEL_BASE. IMHO, the boot code should not search for the '/3GB' option. The kernel base address must be send by freeldr. The boot code must not contain any KERNEL_BASE value. I've add your patch to my source tree. On the smp machine, ros reboots in the very early boot phase. It is before the call to _main.
- Hartmut
Hartmut Birr wrote:
Hi,
I think your patch will never work. There are used some offsets which based on the '/3GB' switch and other ones are hard coded by KERNEL_BASE.
Never? :( Some offsets have to use KERNEL_BASE, and some based on the 3GB switch. I carefully reviewed which were necessary at each point, tho I may have overlooked something. If you like I can go into detail why this is necessary, and why I'm sure it will work.
IMHO, the boot code should not search for the '/3GB' option. The kernel base address must be send by freeldr. The boot code must not contain any KERNEL_BASE value. I've add your patch to my source tree. On the smp machine, ros reboots in the very early boot phase. It is before the call to _main.
Can you shed some light on how exactly app cpus actually get initialized? This will help me to understand more fully what I'm doing.
Hartmut Birr wrote:
Royce Mitchell III schrieb:
I was in a strange mood and decided to try to implement something Alex wanted.
I've done the leg-work on this, but need someone familiar with boot-up and particularly smp to review this patch. I do not have access to smp, and do not wish to break it.
This patch implements the /3G switch from within multiboot.S, which is necessary in order to configure the page tables correctly.
I noticed some code in _main() that was processing a 3G switch on the command-line, and I can't understand how it could possibly work because it's not readjusting the page tables, nor does it transition to the 0xC0000000 address space that I can see, so I don't think that code actually works. If it does, I'm glad to be proved wrong.
The patch I've written makes an assumption that lowmem is available when application processors execute the code. I can't seem to figure out how or where the application processor(s) actually get told to start executing, so I don't know if this assumption is true. However, if it's not true, if someone could point me to the code that inits the application processors, I can configure them to pass the needed info via %ebp or something.
Finally, I haven't actually tried booting this code. I wanted someone to review it first for obvious blunders, or to tell me, if necessary, that I've wasted my time ;) If I'm on the right track then, when I'm feeling motivated enough again, I will actually try booting it.
Ros-dev mailing list Ros-dev@reactos.com http://reactos.com:8080/mailman/listinfo/ros-dev
Hi,
I think your patch will never work. There are used some offsets which based on the '/3GB' switch and other ones are hard coded by KERNEL_BASE. IMHO, the boot code should not search for the '/3GB' option. The kernel base address must be send by freeldr.
I agree...
The boot code must not contain any KERNEL_BASE value. I've add your patch to my source tree. On the smp machine, ros reboots in the very early boot phase. It is before the call to _main.
- Hartmut
I think freeldr shoudl be modified to PE load the boot drivers and ntoskrnl, instead of just memory dumping. This should remove most of the bss hacks which use kernel_base, and the rest should use whatever freeldr pushes on the loader_block. For now, we were simply testing some things, and Royce's patch still helps a lot.
Best regards, Alex Ionescu
I think freeldr shoudl be modified to PE load the boot drivers and ntoskrnl, instead of just memory dumping. This should remove most of the bss hacks which use kernel_base, and the rest should use whatever freeldr pushes on the loader_block. For now, we were simply testing some things, and Royce's patch still helps a lot.
How does windows do this? I gather it does this through NTLDR but what does NTLDR do here? Whatever it is, perhaps we should be doing the same...
Jonathan Wilson wrote:
I think freeldr shoudl be modified to PE load the boot drivers and ntoskrnl, instead of just memory dumping. This should remove most of the bss hacks which use kernel_base, and the rest should use whatever freeldr pushes on the loader_block. For now, we were simply testing some things, and Royce's patch still helps a lot.
How does windows do this? I gather it does this through NTLDR but what does NTLDR do here?
While I haven't really verified this, so I can't be 100% sure, I strongly suspect it does the Right Thing(tm) - PE loads the modules.
This discussion has been going round and round since basically the time I got kernel+hal to compile with MSVC. I had to do some seriously cludgy hackery to
1) "inject" the multiboot header (which is basically just a few bytes telling multiboot "load this image at this hardware address, clean the memory between x and y for BSS use, and then jump to this absolute address afterwards - much like the C64 loaded and ran binary images), and
2) in-place relocate ntoskrnl.exe (this is before paging is turned on, why it was a little tricky to do from a combination of C and inline assembler :-) ). The MinGW build don't have this "problem", since it uses page-sized section alignment (meaning its layout is quite different from a kernel image from an NT system), and depended on GCC-internal stuff to get BSS segment and so on.
The solution is quite obvious: Create a small loader that is multiboot-able (and get loaded by the multiboot-loader loading boot program), that in turn loads the kernel(+hal IIRC) and relocates, makes sure BSS is zero, and jumps to the PE-designated entry-point.
/Mike
Hartmut Birr wrote:
Hi,
I think your patch will never work. There are used some offsets which based on the '/3GB' switch and other ones are hard coded by KERNEL_BASE. IMHO, the boot code should not search for the '/3GB' option. The kernel base address must be send by freeldr. The boot code must not contain any KERNEL_BASE value. I've add your patch to my source tree. On the smp machine, ros reboots in the very early boot phase. It is before the call to _main.
- Hartmut
FWIW, I found a bug in the patch where I was using a MOVL that should have been a LEA. I have stepped through the code in bochs, and it is setting up the page tables correctly. I'm positive we could get this to work if we wanted to. At the moment I'm a bit stuck, because I can't seem to find the freeldr Options string in memory where I think it should be. There's also what appears to be some corrupted pointers or strings in the module list when accessed from _main(). I was tracking it down, but I'm going to put it on hold for reasons I mention below.
Furthermore, at the moment _main() immediately accesses global variables. ntoskrnl would have to reloc itself before we can access any global variables, so the patch will not work as-is.
Finally, there seems to be quite a bit of disagreement about who should relocate ntoskrnl, so I might be wasting my effort trying to get this to actually work, except for the learning value of it all.
Peace,
Royce Mitchell III
Royce Mitchell III wrote:
Finally, there seems to be quite a bit of disagreement about who should relocate ntoskrnl, so I might be wasting my effort trying to get this to actually work, except for the learning value of it all.
I can't see why there should be any disagreement about this. The only logical piece of code to relocate it, is the same piece of code that loads it. Having the kernel relocate itself, while it is running from/in the very image that is to be relocated (!) is not only a bit tricky (http://svn.reactos.com/viewcvs/trunk/msvc6/ntoskrnl/ke_i386_multiboot.c?rev =8046), it's also quite fragile. Should some, any, little piece be changed like code injected or moved to prohibit short function call addressing, it will crash. Should anyone ever make a change in the belief they could access global data (like one usually can from a program), it will crash. Should the very critical link-order change, making the multiboot table i the binary move to outside the first 8192 bytes if the on-disk image, it will crash.
A PE binary normally knows nothing, and shouldn't (have to) know anything, about its on-disk binary stored format. It already has an entry in the PE header telling what/where the entrypoint of the program is. Multiboot on the other hand uses absolute physical addresses, including the entrypoint address (that doesn't even need to point to within the image itself) that it reads from the multiboot-header in the binary. There is no portable way for a PE binary to know at what physical offset from the start of the binary on-disk image a particular piece of code will be without knowing _exactly_ how the linker will place stuff in the on-disk image at compile-time. Linker. Compile-time. Let it sink in a bit.
Having this hard-coded relationship between multiboot and ntoskrnl.exe makes porting harder and the kernel code harder to understand (and therefore maintain).
Should on the other hand the following steps be taken, the ntoskrnl code would become cleaner, more portable and follow logic (that the one doing the loading, also does the relocation) to a much greater degree. The binary image would also no longer contain redundant (and possibly contradictory) information re. where its entry function is, as is the case today.
1. Have a binary that loads the kernel, hal, and the boot drivers into memory. This is done today by freeloader.
2. Have that very same binary do the PE (-section) relocation of these images, including clearing uninitialized data segments as usual for bss (be it explicit as in MinGW built images, or as the "slack" between initialized and virtual size in a/the data section as images built with e.g. MSVC). This is well-known and not very complicated.
3. Have the boot-loader know whether or not /3G is present. This allows _it_ to set up the architecture-specific initial page-table mappings. The benefit of this is, as I see it, that the boot loader is by necessity a very architecture specific piece of code, why it might as well follow what ntldr does. The kernel image would then, when it gets control, already be at the virtual address it's supposed to be, and it can freely access global data as it should have been able to do in the first place. This also has the obvious benefit that much of the _highly_ architecture-specific code can be removed from the kernel itself.
That last point should really be the only argument that's needed.
/Mike
Mike Nordell wrote:
I can't see why there should be any disagreement about this. The only logical piece of code to relocate it, is the same piece of code that loads it. Having the kernel relocate itself, while it is running from/in the very image that is to be relocated (!) is not only a bit tricky (http://svn.reactos.com/viewcvs/trunk/msvc6/ntoskrnl/ke_i386_multiboot.c?rev =8046), it's also quite fragile. Should some, any, little piece be changed like code injected or moved to prohibit short function call addressing, it will crash. Should anyone ever make a change in the belief they could access global data (like one usually can from a program), it will crash. Should the very critical link-order change, making the multiboot table i the binary move to outside the first 8192 bytes if the on-disk image, it will crash.
We can easily make sure that both _main as well as _NtProcessStartup are at the beginning of the file via the linker. This will guarantee we always have short function call addressing. In the new xml build system we can easily mark the files that need to be at the beginning and document *why*.
It will only be a VERY short space of code where global data is inaccessible, and I would make sure that Alex or I document the code sufficiently to illustrate to anybody who cares to look that they can't access global variables there.
Documentation is always the key. Since I've taken the time to study and understand what multiboot.S is actually doing, I can also blog an article about how ntoskrnl begins life ( in addition to the inline documentation ).
A PE binary normally knows nothing, and shouldn't (have to) know anything, about its on-disk binary stored format. It already has an entry in the PE header telling what/where the entrypoint of the program is. Multiboot on the other hand uses absolute physical addresses, including the entrypoint address (that doesn't even need to point to within the image itself) that it reads from the multiboot-header in the binary. There is no portable way for a PE binary to know at what physical offset from the start of the binary on-disk image a particular piece of code will be without knowing _exactly_ how the linker will place stuff in the on-disk image at compile-time. Linker. Compile-time. Let it sink in a bit.
The offsets used at compile time are fixed-up by the linker at link time. I don't see the problem.
Having this hard-coded relationship between multiboot and ntoskrnl.exe makes porting harder and the kernel code harder to understand (and therefore maintain).
This "complexity" is primarily isolated to multiboot.S, except for the self-reloc code that will be called from the beginning of _main(). At least that's the plan. We have two choices here:
1) Do it the way you are suggesting, which is the "clean" way, and presumably the "MS" way, and remain forever indebted to a multiboot compatible proxy loader. This, IMHO, doesn't make anything simpler, it just shuffles the complexity around.
2) Do it in a way that's multiboot compatible, so that we can be loaded by any boot loader people should choose to use. I'd personally much rather choose to be more compatible with FOSS boot loaders at this stage of the OS than with the MS way of doing things. This will make it very easy for dual-booters to use us, which is going to be more important sooner than later. I don't see it having any particularly significant effect on porting complexity. If we were to port ntoskrnl to another archiecture, we'd likely look at how other OS's handle getting multibooted on those architectures. We just duplicate that for that architecture, then call _main() and let it reloc itself in-memory.
- Have the boot-loader know whether or not /3G is present. This allows _it_
to set up the architecture-specific initial page-table mappings. The benefit of this is, as I see it, that the boot loader is by necessity a very architecture specific piece of code, why it might as well follow what ntldr does. The kernel image would then, when it gets control, already be at the virtual address it's supposed to be, and it can freely access global data as it should have been able to do in the first place. This also has the obvious benefit that much of the _highly_ architecture-specific code can be removed from the kernel itself.
That last point should really be the only argument that's needed.
Except that the code is already isolated in the architecture-specific i386 directory. For that reason, I personally don't find it to be a very compelling argument.
All this being said, it was just a curiosity... hacking multiboot.S to support 2G and 3G dynamically only took a couple hours. Fixing ntoskrnl to reloc itself should also be an easy job. It's no skin off my back if people would rather remain forever tied to freeldr. I don't plan to be a dual-booter, and don't imagine after this is said and done that I will ever need to look at this section of code again. It's just that if we're going to discuss changes, we should strongly consider compatibility with the variety of bootloaders that our fellow geeks choose to use.
There's also the "social" reason why we should do it the way I've proposed:
1) the people most likely to complain about having to go through freeldr are people setting up dual-booting. This complaint *will* become a FAQ item, as it will forever come up as an issue.
2) the people most likely to complain about not having the cleaner design of freeldr are the ppl porting to a new architecture. Once it works, they won't complain any more.
There are far fewer new architectures to port ReactOS to than there are ppl who will one day be interested in actually running ReactOS. If we're going to choose a design, we should choose the one that causes the least complaints and "support issues" out of our up-and-coming fan-base.
-- Royce Mitchell III
Hartmut Birr schrieb:
I've add your patch to my source tree. On the smp machine, ros reboots in the very early boot phase. It is before the call to _main.
- Hartmut
Hi,
I've found the problem. MmSystemRangeStart is initialized before the bss section is cleaned. But MmSystemRangeStart is located in the bss section. There are two other little mistakes. The offset of MmSystemRangeStart isn't calculated correctly. Royce has reported this in an other mail. On UP machines MmSystemRangeStart is not initialized. Look at the attached file.
- Hartmut
Hartmut Birr wrote:
I've found the problem. MmSystemRangeStart is initialized before the bss section is cleaned. But MmSystemRangeStart is located in the bss section. There are two other little mistakes. The offset of MmSystemRangeStart isn't calculated correctly. Royce has reported this in an other mail. On UP machines MmSystemRangeStart is not initialized. Look at the attached file.
Thanks Hartmut! I could kick myself for missing that. Easy to fix, I'll give it a try. Want me to send you an updated patch once I do?
Hartmut Birr wrote:
Hartmut Birr schrieb:
I've add your patch to my source tree. On the smp machine, ros reboots in the very early boot phase. It is before the call to _main.
- Hartmut
Hi,
I've found the problem. MmSystemRangeStart is initialized before the bss section is cleaned. But MmSystemRangeStart is located in the bss section. There are two other little mistakes. The offset of MmSystemRangeStart isn't calculated correctly. Royce has reported this in an other mail. On UP machines MmSystemRangeStart is not initialized. Look at the attached file.
Since multiboot loader is supposed to zero bss for us, the simplest solution was to disable the bss zeroing code.
I also found a bug in MmSystemRangeStart was being set at all for UP builds.
It's getting much farther along now when trying to boot. It prints some of the normal ReactOS startup text, but then bugchecks on something, not sure what because the screen gets garbled for some reason.
It still won't fully work yet anyway for at least these two reasons:
1) can't seem to find options string in memory. I know what strings are in my freeldr.ini, and they're just not showing up on the commandline when I step through the code in bochs.
2) ntoskrnl's _main() still needs to reloc itself before it does *anything* else. Otherwise /3GB will bomb.
Royce Mitchell III schrieb:
It still won't fully work yet anyway for at least these two reasons:
- ntoskrnl's _main() still needs to reloc itself before it does
*anything* else. Otherwise /3GB will bomb.
I think we should remove all multiboot code from freeldr and ntoskrnl. Freeldr must relocate ntoskrnl and hal. It must also fix the imports from hal and ntoskrnl. The initial boot code can read the base address from nt header. Freeldr must set the base address within the header.
- Hartmut
-
Alex Ionescu -
Hartmut Birr -
Jonathan Wilson -
Mike Nordell -
Royce Mitchell III