Well, I finally solved my problems with getting freeldr to boot from a hard drive. The first is a problem with fat.bin itself. Early during it's execution it checks to see if the boot sector header specifies a device ID of 0xff, and if so, it uses the device ID the bios passes in CL when it invokes the boot sector, otherwise it uses the device ID in the boot sector header. The device ID field of the boot sector is inherently unreliable and should NEVER be used. In my case mtools was leaving it as it was in fat.bin, which is 0, so it was trying to access the floppy instead of the hard disk. I have fixed this by removing the check to see if the field is 0xff, so the code will always use the device passed by the bios.
The second problem was far more stupid. mtools was not specifying O_BINARY when opening the boot sector template file, and as a result, cygwin was translating newlines, so it clobbered the boot sector.
I'm all set to check in the fix to fat.asm, but I was never asked by subversion ( tortoise client ) to log in, so I assume it just used anon or something. How do I tell subversion to log me in as me?
Phillip Susi wrote:
The second problem was far more stupid. mtools was not specifying O_BINARY when opening the boot sector template file, and as a result, cygwin was translating newlines, so it clobbered the boot sector. I'm all set to check in the fix to fat.asm, but I was never asked by subversion ( tortoise client ) to log in, so I assume it just used anon or something. How do I tell subversion to log me in as me?
tortoise will prompt you for write-enabled credentials the first time you try.
I don't agree with this fix. It removes (valuable) functionality from the boot code and possibly breaks booting on some machines.
There is no spec for how a machine should boot, and the value passed in DL is not standard for all BIOSes. It is basically a left over register value from the MSDOS MBR code. You can't guarantee that it will always be there. I am in charge of the MBR boot code at my day job and I have seen BIOSes which do not pass this value in DL.
Could you please revert the change and fix your value to be either 0xFF or your boot device number (probably 0x80)?
-Brian
-----Original Message----- From: ros-dev-bounces@reactos.com [mailto:ros-dev-bounces@reactos.com] On Behalf Of Phillip Susi Sent: Saturday, April 09, 2005 7:51 PM To: ros-dev@reactos.com Subject: [ros-dev] Freeldr problems resolved
Well, I finally solved my problems with getting freeldr to boot from a hard drive. The first is a problem with fat.bin itself. Early during it's execution it checks to see if the boot sector header specifies a device ID of 0xff, and if so, it uses the device ID the bios passes in CL when it invokes the boot sector, otherwise it uses the device ID in the boot sector header. The device ID field of the boot sector is inherently unreliable and should NEVER be used. In my case mtools was leaving it as it was in fat.bin, which is 0, so it was trying to access the floppy instead of the hard disk. I have fixed this by removing the check to see if the field is 0xff, so the code will always use the device passed by the bios.
The second problem was far more stupid. mtools was not specifying O_BINARY when opening the boot sector template file, and as a result, cygwin was translating newlines, so it clobbered the boot sector.
I'm all set to check in the fix to fat.asm, but I was never asked by subversion ( tortoise client ) to log in, so I assume it just used anon or something. How do I tell subversion to log me in as me?
_______________________________________________ Ros-dev mailing list Ros-dev@reactos.com http://reactos.com:8080/mailman/listinfo/ros-dev
Hrm... I thought it was a standard left over from the PC/AT days at least that the bios would pass the device number in to the boot loader. Otherwise it has no clue from where it is being loaded. For instance, one person could format the disk in their A drive, and the boot sector would get 0 written for the device number, then you try to boot it in the B drive and the boot sector erroneously tries to access the A drive. Likewise for hard disks 80, or 81 and so on, and even when booting from no emulation cdroms, the bios passes in C0 for the first cdrom iirc.
If there are some bioses that do not pass the device number, then setting it to 0xff in freeldr.asm would not fix the problem either, as it would still expect the number from the bios. You would have to manually set it to the right device number for the drive in your system, and only use it in that drive.
Brian Palmer wrote:
I don't agree with this fix. It removes (valuable) functionality from the boot code and possibly breaks booting on some machines.
There is no spec for how a machine should boot, and the value passed in DL is not standard for all BIOSes. It is basically a left over register value from the MSDOS MBR code. You can't guarantee that it will always be there. I am in charge of the MBR boot code at my day job and I have seen BIOSes which do not pass this value in DL.
Could you please revert the change and fix your value to be either 0xFF or your boot device number (probably 0x80)?
-Brian
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160
Phillip Susi wrote:
Hrm... I thought it was a standard left over from the PC/AT days at least that the bios would pass the device number in to the boot loader.
According to the "BIOS Boot Specification," Version 1.01:
"When the boot handler is called, the BIOS passes a pointer to the PnP Installation Check Structure in ES:DI. This is so that once the boot handler has successfully loaded the device?s boot sector into memory at address 0000:7C00h, execution control can be transferred with the following register contents:
ES:DI = Pointer to PnP Installation Check Structure DL = Drive number used for the INT 13h (00h, 80h, etc.)"
So the code follows the specification. The specification is available online at:
http://www.phoenix.com/NR/rdonlyres/56E38DE2-3E6F-4743-835F-B4A53726ABED/0/s...
This is a specification by Compaq Computer Corp., Phoenix Technologies Ltd., and Intel Corp.
If the hardware doesn't support this standard, odds are that nearly nothing will work on it.
- -- Michael B. Trausch fd0man@gmail.com Website: http://fd0man.chadeux.net/ Jabber: mtrausch@jabber.com Phone: +1-(678)-522-7934 FAX (US Only): 1-866-806-4647 =================================================================== Do you have PGP or GPG? Key at pgp.mit.edu, Please Encrypt E-Mail!
That spec is dated 1996. Which is about ~15 years too late.
Actually, most everything will work on hardware that does not support this standard, and pretty much everything that has come out of Microsoft since the beginning. You see, not only is the boot drive value in the BPB of the boot sector reliable, but if it is incorrect no Microsoft OS will be able to boot. That is why the boot drive value exists in the BPB.
When I was writing that boot code I reviewed all the boot sector code I could find on the internet. A lot of the "amateur" boot sector code used DL as the boot drive. But in every instance of commercial boot sector code that I can remember they have never touched DL. At the time I thought that was kind of odd, so I added that code so that both (DL and the BPB boot drive value) could be used. I would bet that if I could go back to when they implemented the boot strap mechanism and look at the code, that the only thing it did was load sector 0 to 0000:7C00 and jmp there. I doubt that any register contents are guaranteed. I said I had seen BIOSes that did not pass the boot drive in DL. I can't remember which exact brand they were, but they were not no-name computers. They were name brand computers like HP or Dell.
In any case, I'm glad you were able to figure out why it wasn't booting. But, I still don't agree with the fact that you removed functionality to fix a corrupt data structure in your boot sector.
-Brian
-----Original Message----- From: ros-dev-bounces@reactos.com [mailto:ros-dev-bounces@reactos.com] On Behalf Of Michael B. Trausch Sent: Sunday, April 10, 2005 11:15 AM To: ReactOS Development List Subject: Re: [ros-dev] Freeldr problems resolved
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160
Phillip Susi wrote:
Hrm... I thought it was a standard left over from the PC/AT days at least that the bios would pass the device number in to the boot loader.
According to the "BIOS Boot Specification," Version 1.01:
"When the boot handler is called, the BIOS passes a pointer to the PnP Installation Check Structure in ES:DI. This is so that once the boot handler has successfully loaded the device?s boot sector into memory at address 0000:7C00h, execution control can be transferred with the following register contents:
ES:DI = Pointer to PnP Installation Check Structure DL = Drive number used for the INT 13h (00h, 80h, etc.)"
So the code follows the specification. The specification is available online at:
http://www.phoenix.com/NR/rdonlyres/56E38DE2-3E6F-4743-835F-B4A53726ABED/0/s pecsbbs101.pdf
This is a specification by Compaq Computer Corp., Phoenix Technologies Ltd., and Intel Corp.
If the hardware doesn't support this standard, odds are that nearly nothing will work on it.
- -- Michael B. Trausch fd0man@gmail.com Website: http://fd0man.chadeux.net/ Jabber: mtrausch@jabber.com Phone: +1-(678)-522-7934 FAX (US Only): 1-866-806-4647 =================================================================== Do you have PGP or GPG? Key at pgp.mit.edu, Please Encrypt E-Mail!
_______________________________________________ Ros-dev mailing list Ros-dev@reactos.com http://reactos.com:8080/mailman/listinfo/ros-dev
Brian Palmer wrote:
That spec is dated 1996. Which is about ~15 years too late.
Yea, so that's 9 years ago. Any computer newer than 9 years old should support it no?
Actually, most everything will work on hardware that does not support this standard, and pretty much everything that has come out of Microsoft since the beginning. You see, not only is the boot drive value in the BPB of the boot sector reliable, but if it is incorrect no Microsoft OS will be able to boot. That is why the boot drive value exists in the BPB.
When I was writing that boot code I reviewed all the boot sector code I could find on the internet. A lot of the "amateur" boot sector code used DL as the boot drive. But in every instance of commercial boot sector code that I can remember they have never touched DL. At the time I thought that was kind of odd, so I added that code so that both (DL and the BPB boot drive value) could be used. I would bet that if I could go back to when they implemented the boot strap mechanism and look at the code, that the only thing it did was load sector 0 to 0000:7C00 and jmp there. I doubt that any register contents are guaranteed. I said I had seen BIOSes that did not pass the boot drive in DL. I can't remember which exact brand they were, but they were not no-name computers. They were name brand computers like HP or Dell.
Were any of them newer than 1996? If it is only computers older than 1996 that we have to worry about, I don't think that is much of a concern.
Anyhow, just changing the value to 0xff would not have any better effect, as it would still be looking to DL even on computers that do not set it, so how is that any better of a fix?
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160
Phillip Susi wrote:
Yea, so that's 9 years ago. Any computer newer than 9 years old should support it no?
That's what I was thinking. Besides, some of the boot code that I've written myself depends on that behavior (and I wrote it that way specifically because there is a multi-vendor standard to back it up). I have yet to find a computer that doesn't boot it off of any boot device.
Just because all of Microsoft's filesystems use a BPB doesn't mean that *every single* device does. Under Windows NT raw hardware access is denied, anyway, just like any other real 32-bit protected system should be. Meaning that that data should only be there if it is of real benefit to the operating system, and if not, that's several bytes that can be used by the first stage boot loader for whatever purposes (intelligent finding of a system file to load, perhaps, so that the kernel can be dynamically located somewhere in the root directory, verses a boot-block-list being required or something like that).
I am sure that while Microsoft systems aren't the only family to use the BPB method of doing things, that there are plenty of other systems out there that don't use that method for one reason or another. I think it's more beneficial to have a static boot sector rather then one that needs to be updated constantly by programs that format or initialize a filesystem.
Perhaps when I have a little bit more time I'll look into the source for some other programs and see how they manage the task of being on bootable media and booting. However, as far as I can tell from what I've looked at so far, LILO doesn't use a BPB, just taking a first glance at their assembly language source to the boot loader.
Sure, it'd be wise to support the BPB for FATxx file systems as well as NTFS, but not because it's "generally a smarter thing to do." More like because that's the published standard for those filesystems, so any program interoperating with those particular filesystems should adhere to that standard. But does that mean that ReactOS is bound to it? Certainly not. If the ReactOS programmers go and implement support in the kernel for ext3 and ReiserFS, and they can use that as the system boot volume's filesystem, then most certainly their boot loader shouldn't be tied to objects from another family of filesystems.
A quick (*NOT* complete - so don't take this as etched in stone) glance on Google tells me that there isn't a specified format for boot sectors for ext[23] or ReiserFS, meaning that nobody should rely on them for information about the filesystem or boot device, but rather that the operating system should be able to figure that out on it's own.
I think that the standard mentioned in my last post was probably created to help define the state of the system to promote alternative operating systems growth, without hindering their ability to figure out what's going on with the computer. That's a Good Thing, IMHO.
Were any of them newer than 1996? If it is only computers older than 1996 that we have to worry about, I don't think that is much of a concern.
Anyhow, just changing the value to 0xff would not have any better effect, as it would still be looking to DL even on computers that do not set it, so how is that any better of a fix?
I personally have yet to find a system that does not place the value of the boot device into the DL register. I'm not an expert -- or even an intermediate level -- programmer by any stretch of the imagination, but for the various hobbyist level type things that I've tinkered with, all of it has worked on every x86 box I've had my hands on -- mind you, nothing older then probably 1998 or so. I only started seriously getting into computers and different operating systems in 1996, and from there, started programming in probably 1999 or so, and by the time I started tinkering with OS-level things, it was 2001 and I had nothing older then 1998 lying around. So we're talking a total of maybe 30 - 40 machines, but all of them were able to boot my code, and all of them were from various manufacturers with different BIOS vendors.
- Mike
- -- Michael B. Trausch fd0man@gmail.com Website: http://fd0man.chadeux.net/ Jabber: mtrausch@jabber.com Phone: +1-(678)-522-7934 FAX (US Only): 1-866-806-4647 =================================================================== Do you have PGP or GPG? Key at pgp.mit.edu, Please Encrypt E-Mail!
I will try to find out what computer system it was that had the BIOS that didn't pass the boot drive in DL. I believe that GRUB also supports a similar mechanism for BIOSes that don't pass the boot drive in DL.
As far as being able to boot from EXT2 - FreeLoader has supported that for many years now. It can also boot Linux kernels natively. There are also other flag bytes in the FreeLoader boot sector code that help out with determining which partition FreeLoader is installed on.
-Brian
-----Original Message----- From: ros-dev-bounces@reactos.com [mailto:ros-dev-bounces@reactos.com] On Behalf Of Michael B. Trausch Sent: Sunday, April 10, 2005 2:41 PM To: ReactOS Development List Subject: Re: [ros-dev] Freeldr problems resolved
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160
Phillip Susi wrote:
Yea, so that's 9 years ago. Any computer newer than 9 years old should support it no?
That's what I was thinking. Besides, some of the boot code that I've written myself depends on that behavior (and I wrote it that way specifically because there is a multi-vendor standard to back it up). I have yet to find a computer that doesn't boot it off of any boot device.
Just because all of Microsoft's filesystems use a BPB doesn't mean that *every single* device does. Under Windows NT raw hardware access is denied, anyway, just like any other real 32-bit protected system should be. Meaning that that data should only be there if it is of real benefit to the operating system, and if not, that's several bytes that can be used by the first stage boot loader for whatever purposes (intelligent finding of a system file to load, perhaps, so that the kernel can be dynamically located somewhere in the root directory, verses a boot-block-list being required or something like that).
I am sure that while Microsoft systems aren't the only family to use the BPB method of doing things, that there are plenty of other systems out there that don't use that method for one reason or another. I think it's more beneficial to have a static boot sector rather then one that needs to be updated constantly by programs that format or initialize a filesystem.
Perhaps when I have a little bit more time I'll look into the source for some other programs and see how they manage the task of being on bootable media and booting. However, as far as I can tell from what I've looked at so far, LILO doesn't use a BPB, just taking a first glance at their assembly language source to the boot loader.
Sure, it'd be wise to support the BPB for FATxx file systems as well as NTFS, but not because it's "generally a smarter thing to do." More like because that's the published standard for those filesystems, so any program interoperating with those particular filesystems should adhere to that standard. But does that mean that ReactOS is bound to it? Certainly not. If the ReactOS programmers go and implement support in the kernel for ext3 and ReiserFS, and they can use that as the system boot volume's filesystem, then most certainly their boot loader shouldn't be tied to objects from another family of filesystems.
A quick (*NOT* complete - so don't take this as etched in stone) glance on Google tells me that there isn't a specified format for boot sectors for ext[23] or ReiserFS, meaning that nobody should rely on them for information about the filesystem or boot device, but rather that the operating system should be able to figure that out on it's own.
I think that the standard mentioned in my last post was probably created to help define the state of the system to promote alternative operating systems growth, without hindering their ability to figure out what's going on with the computer. That's a Good Thing, IMHO.
Were any of them newer than 1996? If it is only computers older than 1996 that we have to worry about, I don't think that is much of a concern.
Anyhow, just changing the value to 0xff would not have any better effect, as it would still be looking to DL even on computers that do not set it, so how is that any better of a fix?
I personally have yet to find a system that does not place the value of the boot device into the DL register. I'm not an expert -- or even an intermediate level -- programmer by any stretch of the imagination, but for the various hobbyist level type things that I've tinkered with, all of it has worked on every x86 box I've had my hands on -- mind you, nothing older then probably 1998 or so. I only started seriously getting into computers and different operating systems in 1996, and from there, started programming in probably 1999 or so, and by the time I started tinkering with OS-level things, it was 2001 and I had nothing older then 1998 lying around. So we're talking a total of maybe 30 - 40 machines, but all of them were able to boot my code, and all of them were from various manufacturers with different BIOS vendors.
- Mike
- -- Michael B. Trausch fd0man@gmail.com Website: http://fd0man.chadeux.net/ Jabber: mtrausch@jabber.com Phone: +1-(678)-522-7934 FAX (US Only): 1-866-806-4647 =================================================================== Do you have PGP or GPG? Key at pgp.mit.edu, Please Encrypt E-Mail!
_______________________________________________ Ros-dev mailing list Ros-dev@reactos.com http://reactos.com:8080/mailman/listinfo/ros-dev
Yea, so that's 9 years ago. Any computer newer than 9 years old should support it no? Were any of them newer than 1996? If it is only computers older than 1996 that we have to worry about, I don't think that is much of a concern.
Well I fixed this bug (DL not passed in by BIOS) about a year and a half ago. So, assuming that the buggy computer was a year and a half old at the time that gives us a 3 year old computer that FreeLoader might have problems with.
Anyhow, just changing the value to 0xff would not have any better effect, as it would still be looking to DL even on computers that do not set it, so how is that any better of a fix?
This seems like a very closed minded statement to me. How can options and configurability be any worse that having no choice at all? The whole idea behind that code was to use the value in the BPB (because it SHOULD always be correct for the disk), but, optionally, if you wanted to use the value from the DL register you could do that also by setting the boot drive in the BPB to 0xFF.
Now I'm not saying the using the boot drive in the BPB is the 'right' way, and I'm also not saying that using DL is the 'right' way. But, if that code is removed then if you don't get the boot drive in DL you will not be able to boot, and there is nothing you can do about it. Except add the code back in. I don't know about you, but I'd like to have the choice.
-Brian
_______________________________________________ Ros-dev mailing list Ros-dev@reactos.com http://reactos.com:8080/mailman/listinfo/ros-dev
Hrm... ok, how about this then. The code should go back to check if it's 0xff and use DL if it is, the default value in fat.asm should be set to 0xff, but then who is ever going to set it to the 'correct' value on systems that don't pass the right value in dl?
It would be freeldr's installer's job to do that. When it installs the boot sector it should probably leave it as 0xff by default, but give the user the option of changing it to a value they specify, and suggest they try 80 for hard drives and 0 for floppies if their busted bios can't boot it with ff.
Brian Palmer wrote:
Yea, so that's 9 years ago. Any computer newer than 9 years old should support it no? Were any of them newer than 1996? If it is only computers older than 1996 that we have to worry about, I don't think that is much of a concern.
Well I fixed this bug (DL not passed in by BIOS) about a year and a half ago. So, assuming that the buggy computer was a year and a half old at the time that gives us a 3 year old computer that FreeLoader might have problems with.
Anyhow, just changing the value to 0xff would not have any better effect, as it would still be looking to DL even on computers that do not set it, so how is that any better of a fix?
This seems like a very closed minded statement to me. How can options and configurability be any worse that having no choice at all? The whole idea behind that code was to use the value in the BPB (because it SHOULD always be correct for the disk), but, optionally, if you wanted to use the value from the DL register you could do that also by setting the boot drive in the BPB to 0xFF.
Now I'm not saying the using the boot drive in the BPB is the 'right' way, and I'm also not saying that using DL is the 'right' way. But, if that code is removed then if you don't get the boot drive in DL you will not be able to boot, and there is nothing you can do about it. Except add the code back in. I don't know about you, but I'd like to have the choice.
-Brian
Sounds like a good solution to me.
-Brian
-----Original Message----- From: ros-dev-bounces@reactos.com [mailto:ros-dev-bounces@reactos.com] On Behalf Of Phillip Susi Sent: Sunday, April 10, 2005 4:08 PM To: ReactOS Development List Subject: Re: [ros-dev] Freeldr problems resolved
Hrm... ok, how about this then. The code should go back to check if it's 0xff and use DL if it is, the default value in fat.asm should be set to 0xff, but then who is ever going to set it to the 'correct' value on systems that don't pass the right value in dl?
It would be freeldr's installer's job to do that. When it installs the boot sector it should probably leave it as 0xff by default, but give the user the option of changing it to a value they specify, and suggest they try 80 for hard drives and 0 for floppies if their busted bios can't boot it with ff.
Brian Palmer wrote:
Yea, so that's 9 years ago. Any computer newer than 9 years old should support it no? Were any of them newer than 1996? If it is only computers older than 1996 that we have to worry about, I don't think that is much of a concern.
Well I fixed this bug (DL not passed in by BIOS) about a year and a half ago. So, assuming that the buggy computer was a year and a half old at the time that gives us a 3 year old computer that FreeLoader might have
problems
with.
Anyhow, just changing the value to 0xff would not have any better effect, as it would still be looking to DL even on computers that do not set it, so how is that any better of a fix?
This seems like a very closed minded statement to me. How can options and configurability be any worse that having no choice at all? The whole idea behind that code was to use the value in the BPB (because it SHOULD always be correct for the disk), but, optionally, if you wanted to use the value from the DL register you could do that also by setting the boot drive in
the
BPB to 0xFF.
Now I'm not saying the using the boot drive in the BPB is the 'right' way, and I'm also not saying that using DL is the 'right' way. But, if that code is removed then if you don't get the boot drive in DL you will not be able to boot, and there is nothing you can do about it. Except add the code back in. I don't know about you, but I'd like to have the choice.
-Brian
_______________________________________________ Ros-dev mailing list Ros-dev@reactos.com http://reactos.com:8080/mailman/listinfo/ros-dev
-
Brian Palmer -
Michael B. Trausch -
Phillip Susi -
Royce Mitchell III