I was thinking about the security policy and NTFS partitions, I really like this function, but this is only availible on NTFS, but why not make the FAT driver head the policies and make it work as NTFS?
Of course it's really easy to boot from a DOS disk and be able to read/write any file, but being able to set access policys on a home-computer/network is a real help for an administrator. It doesn't stop a hacker, but children and wifes (some of them anyway).
Microsoft isn't interested in such a driver because they want everyone to run NTFS, but we don't have such preference, we are interested in the best function not our habitat.
Yours sincerely, Jaix Bly
reactos wrote:
I was thinking about the security policy and NTFS partitions, I really like this function, but this is only availible on NTFS, but why not make the FAT driver head the policies and make it work as NTFS?
No... it is also present on most GNU/Linux filesystems, like ext3...
João Jerónimo
Hi! João Jerónimo Barata de Oliveira wrote:
reactos wrote:
I was thinking about the security policy and NTFS partitions, I really like this function, but this is only availible on NTFS, but why not make the FAT driver head the policies and make it work as NTFS?
No... it is also present on most GNU/Linux filesystems, like ext3...
João Jerónimo
Let us merge the Ext2 ros branch to head and start fixing it! Once that is done we can move to ext3 variation.
James
James Tabor wrote:
I was thinking about the security policy and NTFS partitions, I really like this function, but this is only availible on NTFS, but why not make the FAT driver head the policies and make it work as NTFS?
No... it is also present on most GNU/Linux filesystems, like ext3...
Let us merge the Ext2 ros branch to head and start fixing it! Once that is done we can move to ext3 variation.
LOL...
I was not sugesting such a thing!
João Jerónimo
Hi! There are patent issues here.
João Jerónimo Barata de Oliveira wrote:
James Tabor wrote:
I was thinking about the security policy and NTFS partitions, I really like this function, but this is only availible on NTFS, but why not make the FAT driver head the policies and make it work as NTFS?
No... it is also present on most GNU/Linux filesystems, like ext3...
Let us merge the Ext2 ros branch to head and start fixing it! Once that is done we can move to ext3 variation.
LOL...
I was not sugesting such a thing!
João Jerónimo
Using ntfs as anything other than just coping files to and from, will not happen! ReactOS can not use ntfs as an backbone file system. We are getting away with it just using vfat. So,,, all the wasted band width over ntfs!?
Why bother, make ext2 work, and that is what I'm doing atm, James
Why do you say full use of NTFS will not happen? I believe it should and needs to happen. Unless you find a way to hack ext2 so that it can store EAs, named data streams, and real NT security descriptors instead of the weak file mode bits. Oh, and make it journaling.
Simply put, ext2 does not satisfy the needs of an NT filesystem, so you can either overhaul it so that it does, or just work with the existing NTFS filesystem.
James Tabor wrote:
Using ntfs as anything other than just coping files to and from, will not happen! ReactOS can not use ntfs as an backbone file system. We are getting away with it just using vfat. So,,, all the wasted band width over ntfs!?
Why bother, make ext2 work, and that is what I'm doing atm, James _______________________________________________
Philip: Any filesystem is plenty insecure if you are accessing it via a bootable CD-ROM or other such measures. Also, ext2 already has a journal, it's called ext3.
Yes, a filesystem is only as secure as the operating system accessing it, but that does not change the fact that the filesystem needs to store proper security descriptors, which ext2/3 can not do AFAIK. It also can't do things like store EAs and named data streams, also, AFAIK. If it can do these things, then maybe we could use it as the filesystem of choice, but otherwise, it is either NTFS, or design a whole new filesystem with all the needed features.
Mike Swanson wrote:
Philip: Any filesystem is plenty insecure if you are accessing it via a bootable CD-ROM or other such measures. Also, ext2 already has a journal, it's called ext3.
James Tabor wrote:
Why do you say full use of NTFS will not happen?
I thought we should stay away from NTFS. But if I'm wrong, that's okay with me.
James
NTFS is (and will probably remain) not-so-easy to implement, because it's not terribly well documented. While it might be good to be able to read the filesystems after the system is fully stable and such, it might not be a good idea to attempt to fully implement it.
However, on the other side of that, is this: NTFS is just plain *horrible* if you try to gracefully resize it. Very rarely have I heard of an NTFS-resizing operation come out without some form of data loss or filesystem corruption.
That having been said, ReactOS can't possibly expect to be able to dual-boot and subsequently replace Windows on a single partition system. It's hard enough to try to convince Linux to boot in a scenerio where 100% of a hard disk is NTFS and a backup, repartition, reformat, restore is not an option.
I do think, though, that in the long run, it's safer to just avoid NTFS altogether. It's a huge minefield of messy destruction that could probably wait until after the rest of the system has matured, to be attacked. I'd rather move people away from NTFS and closer to something like ext3 with advanced ACLs. EXT3 at the very least is better then something like NTFS, because it's open, performs decently well, and has support for many different things that NTFS does not. I think it'd be great to have ReactOS have support for it and all of it's features, natively, taking advantage of things like the "executable" attribute that you give a file, instead of the Windows philosophy that everything that's .exe, .cmd, .vbs, .bat, etc., be executable by default.
Or functionality that when the user (or program on the user's behalf) attempted to execute a file, would check the bit and if it wasn't set, prompt the user if they'd like to have it set. This could easily thwart systems that rely on blindly executing (such as many of the worms and viruses that there are in today's world) from ever seeing the light of day, if the system doesn't execute it by default - similar to the approach used by firewall systems that prohibit applications and other programs from accessing the Internet without first being approved by the user (things like Windows XP SP2's firewall, and ZoneLabs Integrity Client, do this).
Anyway, I'll hop off of my soapbox. :-P
- Mike
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Michael B. Trausch wrote:
I think it'd be great to have ReactOS have support for it and all of it's features, natively, taking advantage of things like the "executable" attribute that you give a file, instead of the Windows philosophy that everything that's .exe, .cmd, .vbs, .bat, etc., be executable by default.
True, but then you'd get things like "weekend.jpg" with the X bit set and that'd cause huge problems.
-K
Michael B. Trausch wrote:
NTFS is (and will probably remain) not-so-easy to implement, because it's not terribly well documented. While it might be good to be able to read the filesystems after the system is fully stable and such, it might not be a good idea to attempt to fully implement it.
It may be easier to implement our own filesystem, I'm not sure. At this point I kind of doubt it as I believe the linux ntfs project has documented NTFS nearly completely. I also believe that neither fat nor ext2/3 satisfies our needs with regards to things like EAs, named data streams, and security descriptors.
However, on the other side of that, is this: NTFS is just plain *horrible* if you try to gracefully resize it. Very rarely have I heard of an NTFS-resizing operation come out without some form of data loss or filesystem corruption.
Resize using what utility? Resizing a filesystem is not a function of the filesystem itself, but the software used to rebuild it. The actual NTFS format itself actually lends itself quite well to dynamic resizing. All that is required is to extend/retract the $Bitmap file accordingly. I myself have used Norton Ghost many times to create a system image and later restore that image to several PCs with larger hard drives than the original. Ghost handled resizing the partition flawlessly. Partition Magic on the other hand, I have seen trash hard drives several times.
That having been said, ReactOS can't possibly expect to be able to dual-boot and subsequently replace Windows on a single partition system. It's hard enough to try to convince Linux to boot in a scenerio where 100% of a hard disk is NTFS and a backup, repartition, reformat, restore is not an option.
That is because Linux inherently does not like NTFS. As an NT clone, ReactOS does. Provided that we implement an NTFS driver, installing reactos on an existing windows partition and either dual booting or replacing windows would be quite feasible.
I do think, though, that in the long run, it's safer to just avoid NTFS altogether. It's a huge minefield of messy destruction that could probably wait until after the rest of the system has matured, to be attacked. I'd rather move people away from NTFS and closer to something like ext3 with advanced ACLs. EXT3 at the very least is better then something like NTFS, because it's open, performs decently well, and has support for many different things that NTFS does not. I think it'd be great to have ReactOS have support for it and all of it's features, natively, taking advantage of things like the "executable" attribute that you give a file, instead of the Windows philosophy that everything that's .exe, .cmd, .vbs, .bat, etc., be executable by default.
What features does ext3 support? At the moment I only know of features it does NOT support that NTFS does, and we will need.
Or functionality that when the user (or program on the user's behalf) attempted to execute a file, would check the bit and if it wasn't set, prompt the user if they'd like to have it set. This could easily thwart systems that rely on blindly executing (such as many of the worms and viruses that there are in today's world) from ever seeing the light of day, if the system doesn't execute it by default - similar to the approach used by firewall systems that prohibit applications and other programs from accessing the Internet without first being approved by the user (things like Windows XP SP2's firewall, and ZoneLabs Integrity Client, do this).
Anyway, I'll hop off of my soapbox. :-P
- Mike
That is an application issue really. Specifically web browsers and email clients that blindly ShellExecute() files that come from unknown sources. They usually DO warn the user, but users usually ignore the warning. That is why system administrators should configure the machine to not execute untrusted programs.
What features does ext3 support? At the moment I only know of features it does NOT support that NTFS does, and we will need.
I rather begin to think that there are features you dont want ext? to support it. What-o-ever. This link will answer your questions: http://en.wikipedia.org/wiki/Comparison_of_file_systems
Actually ext3 and my preffered Fsys JFS do support EA. However there is seemingly some difference between OS2-Ea and posix-EA. Thus JFS coming from OS/2, it would fit best. Of course all my endeavours ended in nothing.
I share the opinion that NFFS is a thing we should more less avoid. People are happy if one fs provides reliable access and if it is one which is also accessible by linux, so what's better? ReactOS users will potentially avoid MS-NT
So coming back to ext?. There exists a rather good implementation for WinNT: http://www.fs-driver.org/index.html better than the italian one. So what's the source of ROS's ext2-IFS in SVN? Did we already contact the programmer of the mentioned ext2-ifs? Maybe he is willing to add ACL and owner-support and a Posix-EA <-> IBM-EA mapping to his implementation and work with us?
Issues: Owners are 16-bit (since 2.6 also 32-Bit) integers. On NT they are GUIDs. There is some partitioning in DOMAIN and USER for the GUID. However noone hinders us to add a flag to our SM to just use 16 or 32- bit user part for GUIDs to have an easy mapping (also on Ext2,UFS,Reiser...)
Multiple File streams. NTFS is (looking at the mentioned table) the only FS capable of that feature. But what is it? It is just some semantic to actually hide files having a colon it their name (more less). So such feature can be easyly simulated by a ROS-IFS. OK, there are issues with attributes which then exist several times. But this is in my opinion a minor problem, since no corruption by other OSs can occour (as in FAT). And be honest. Who uses this feature? MS' AppleTalk-Server AND some Virus that hides in another stream.
Robert Köpferl wrote:
I share the opinion that NFFS is a thing we should more less avoid. People are happy if one fs provides reliable access and if it is one which is also accessible by linux, so what's better? ReactOS users will potentially avoid MS-NT
Although I think it is of less importance, I can not disagree on this point more. Most users who will be switching to ReactOS will switching from windows, not Linux, and so they will be concerned about NTFS compatibility, not ext. NTFS compatibility will be a desirable feature for most ReactOS users, but I see it as a nice feature, but not vitally important. The important reason for using NTFS as I see it, is that I do not think other filesystems support the needed features.
So coming back to ext?. There exists a rather good implementation for WinNT: http://www.fs-driver.org/index.html better than the italian one. So what's the source of ROS's ext2-IFS in SVN? Did we already contact the programmer of the mentioned ext2-ifs? Maybe he is willing to add ACL and owner-support and a Posix-EA <-> IBM-EA mapping to his implementation and work with us?
To my knowledge, ext has no way to store NT security descriptors. You may be able to write a hack that say, stores all the descriptors in one file, and then has to search that file for the corresponding descriptor for every other file on the volume you try to access, but that is an ugly hack.
Issues: Owners are 16-bit (since 2.6 also 32-Bit) integers. On NT they are GUIDs. There is some partitioning in DOMAIN and USER for the GUID. However noone hinders us to add a flag to our SM to just use 16 or 32- bit user part for GUIDs to have an easy mapping (also on Ext2,UFS,Reiser...)
User IDs on NT are SIDs, which are not the same as GUIDs. Technically SIDs can have variable length, but all MS windows implementations to date have used a 128 bit length consisting of a domain specific part and a user specific part. Mapping this structure back and forth to a 32 bit posix uid is an ugly hack, and as such, is less desirable than native support. The same goes for trying to map NT ACLs back and forth to posix chmod bits.
Multiple File streams. NTFS is (looking at the mentioned table) the only FS capable of that feature. But what is it? It is just some semantic to actually hide files having a colon it their name (more less). So such feature can be easyly simulated by a ROS-IFS. OK, there are issues with attributes which then exist several times. But this is in my opinion a minor problem, since no corruption by other OSs can occour (as in FAT). And be honest. Who uses this feature? MS' AppleTalk-Server AND some Virus that hides in another stream.
A number of features use data streams including offline files, EFS, on the fly compressed files, and the summary property sheet that lets users associate information with files including the author, revision, description, and search keywords. There is also the file OBJID feature and reparse points. I am sure there are more things that rely on this feature that I am forgetting about, and I am also sure that there are several nice features that we could implement in the future using streams, so yes, it is something that we want to support.
In the end, you may be able to hack all the features of NTFS into some other filesystem, but it will take a good amount of dirty tricks and so won't perform nearly as well. It is better to go with a filesystem that was designed to handle these features rather than come up with kludgey ways to force the features into some other filesystem that was not.
Phillip Susi wrote:
Robert Köpferl wrote:
So coming back to ext?. There exists a rather good implementation for WinNT: http://www.fs-driver.org/index.html better than the italian one. So what's the source of ROS's ext2-IFS in SVN? Did we already contact the programmer of the mentioned ext2-ifs? Maybe he is willing to add ACL and owner-support and a Posix-EA <-> IBM-EA mapping to his implementation and work with us?
To my knowledge, ext has no way to store NT security descriptors. You may be able to write a hack that say, stores all the descriptors in one file, and then has to search that file for the corresponding descriptor for every other file on the volume you try to access, but that is an ugly hack.
You can do it the same way Samba does it by using xattrs. It's a shame that so few ReactOS developers turned up to WineConf as the Samba team did a presentation on this.
Multiple File streams. NTFS is (looking at the mentioned table) the only FS capable of that feature. But what is it? It is just some semantic to actually hide files having a colon it their name (more less). So such feature can be easyly simulated by a ROS-IFS. OK, there are issues with attributes which then exist several times. But this is in my opinion a minor problem, since no corruption by other OSs can occour (as in FAT). And be honest. Who uses this feature? MS' AppleTalk-Server AND some Virus that hides in another stream.
A number of features use data streams including offline files, EFS, on the fly compressed files, and the summary property sheet that lets users associate information with files including the author, revision, description, and search keywords.
Again, you can use xattrs.
There is also the file OBJID feature and reparse points. I am sure there are more things that rely on this feature that I am forgetting about, and I am also sure that there are several nice features that we could implement in the future using streams, so yes, it is something that we want to support.
Now you're scraping the barrel.
Rob
Robert Shearman wrote:
Phillip Susi wrote:
[snip]
There is also the file OBJID feature and reparse points.
[snip]
Now you're scraping the barrel.
Reparse points is a feature of NTFS I've myself used since over half a decade now. As I know I'm not alone using both junction points and volume mount points, I'd think scraping the barrel might be "somewhat" misleading.
/Mike
How exactly are xattrs stored on disk? Is it possible and efficient to store a dozen xattrs on a file of various sizes from 1kb to several MB? How would you reconcile the two different sets of owner and access information associated with the file? For instance, if someone creates a file on the partition from Linux, then you mount it with ROS, who owns the file? Who has access to it? Linux would only place some meaningless uid and chmod mask in the inode, so how would ROS come up with a sensible security descriptor for the file when one does not already exist?
These are the kinds of problems that make attempts for ReactOS to share a unix filesystem ( and vice versa ) kludgey at best.
Robert Shearman wrote:
You can do it the same way Samba does it by using xattrs. It's a shame that so few ReactOS developers turned up to WineConf as the Samba team did a presentation on this.
Again, you can use xattrs.
Phillip Susi wrote:
How exactly are xattrs stored on disk? Is it possible and efficient to store a dozen xattrs on a file of various sizes from 1kb to several MB? How would you reconcile the two different sets of owner and access information associated with the file? For instance, if someone creates a file on the partition from Linux, then you mount it with ROS, who owns the file? Who has access to it? Linux would only place some meaningless uid and chmod mask in the inode, so how would ROS come up with a sensible security descriptor for the file when one does not already exist?
These are the kinds of problems that make attempts for ReactOS to share a unix filesystem ( and vice versa ) kludgey at best.
These are acutally problems you run into if you even try to share an NTFS filesystem. The problem is that it is just the unit of Disk-FS and OS with its conf-data. As you know: Not the names of groups and users are signifficant, the SIDs are. And SIDs get createt randomly. So the actual intend is rather to make use of a JFS, ext2, reiser, Bfs, ... in terms of NT-ifying it and use it as far as its features allow. Even FAT is still good enough to install NT. However signifficant features are missing. I've seen some NTFS and ext2-partitions that are used for 'storrage', shared storrage. They're all r/w to everyone... :-/
Phillip Susi wrote:
How exactly are xattrs stored on disk? Is it possible and efficient to store a dozen xattrs on a file of various sizes from 1kb to several MB? How would you reconcile the two different sets of owner and access information associated with the file? For instance, if someone creates a file on the partition from Linux, then you mount it with ROS, who owns the file? Who has access to it? Linux would only place some meaningless uid and chmod mask in the inode, so how would ROS come up with a sensible security descriptor for the file when one does not already exist?
These are the kinds of problems that make attempts for ReactOS to share a unix filesystem ( and vice versa ) kludgey at best.
You get the same problems when sharing an NTFS filesystem on a dual-boot system or when moving one between computers, unless the SIDs are well-known. I would suggest you check out the Samba presentation at the following URL for information on how Samba4 does NTFS->Unix permissions conversion so you are not stabbing in the dark: http://wiki.winehq.org/WineConf
The problem you are describing has already been solved by them in a slightly-not-kludgey way.
Rob
Phillip Susi wrote:
These are the kinds of problems that make attempts for ReactOS to share a unix filesystem ( and vice versa ) kludgey at best.
It seems MS engineers already solved the problem. Try installing the full SFU suite (NIS/AD bridge, NFS server etc.) on a Windows Server 2003 DC and look at a file's properties...
ea wrote:
Phillip Susi wrote:
These are the kinds of problems that make attempts for ReactOS to share a unix filesystem ( and vice versa ) kludgey at best.
It seems MS engineers already solved the problem. Try installing the full SFU suite (NIS/AD bridge, NFS server etc.) on a Windows Server 2003 DC and look at a file's properties...
AND? What? Tell! Sorry that I am currently too lazy to do that....
Ros-dev mailing list Ros-dev@reactos.com http://reactos.com:8080/mailman/listinfo/ros-dev
I have intentionally stayed out of this discussion, since it seems it's much about personal preferences and less about what I believe initially likely matters more for ROS - a way for the users coming to ROS to continue to use their existing data.
I believe that for this argument to be constructive, first it has to be decided what users are most important, or even likely, to be able to see the priority of FS implementations: - New users that have no previous data that matters to them. - Users migrating from Windows, currently using NTFS. - Users migrating from unix-like operating systems.
Considering ROS will be a way for people to get out of the Microsoft treadmill while continue to use a familiar environment (and for developers, API) it would seem like a logical conclusion that the majority already have filesystems usable from Windows. Unless those users are coming from a Win9x environment, chances are it is NTFS.
I personally know of people willing to get away from Microsoft but having terabytes of data that would be unaccessible from ROS should an NTFS driver not be implemented, making those potential users not even bother to sneeze. Even if given a path to migrate that data to a filesystem supported by ROS; would _you_ want to migrate one or more terabyte(s) of data from something proven reliable (so far) to something almost unknown? Would you do it if you suspected it was a one-way trip? I sure know I wouldn't, and neither would they.
Robert Köpferl wrote:
So coming back to ext?. There exists a rather good implementation for WinNT:
I would say "rather good" is nowhere near the rock solid stability required for a filesystem. The filesystem is _the_ most important piece for a system (and its users), since it is what is supposed to hold and safeguard its data. Everything, and I do mean everything else can be replaced, but some data is unique and cant be replaced (start rant about backup...). Applying the often disliked Microsoft-mantra of "good enough" in this area seems more like asking users to become part of the Shroedinger experiment - only catch is that the users data would be the cat.
Multiple File streams. NTFS is (looking at the mentioned table) the only FS capable of that feature.
Actually, that's also not entirely true. BFS, the filesystem for BeOS also had/has it (*), though much better integrated into the OS and tools (of course?), and... didn't Apple HFS use a different stream for the resource fork?
(*) In form of named properties. To be even more correct, BFS named attributes are at the FS level more like NTFS' attributes, even that they can also store data much like a named data stream in NTFS.
Taking a little detour: While multiple (named) streams display the complete incomptibility with most other filesystems in the world, at least if you want to move a file using Z-Modem or something similarly primitive, I have yet to decide which way is the best for general use - using but a single data stream for simplicity and compatibility with systems back from BC, or go for multiple streams for expressive power.
Having used NTFS, where this is very poorly integrated (when at all) into the OS, API and tools, and BFS where the very good OS integration made it feel just "natural", I suspect multiple streams indeed is the better way, but it can't be used to a fraction of its potential with the primitive API's or API semantics used since BC (this essentially includes the Win32 API and tools).
While the rest of the computing industry have moved quite a bit in the last 25+ years, FS semantics and API's have esentially been standing still. Having used "something better" (my subjective opinion based on BFS+BeOS), I'd say this standstill is not because the current paradigm is good enough. Compared to just that "something better" I'd say current paradigm sucks.
Even more OT opinions: While it would be really nice to lift this up to at least the level of the BeOS+BFS functionality, this is not the place. At least not yet. Once ROS is more mature and complete I wouldn't mind see this added for a new filesystem, making ROS take a leap. But that's currently so far into the future that raising the question now is really pointless. There are lots and lots of grunt-work more important before starting to add features and extending API's, filesystems and network protocols, making ROS apps taking advantage of these improvements incompatible with Win32/NT.
/Mike
Mike Nordell wrote:
I have intentionally stayed out of this discussion, since it seems it's much about personal preferences and less about what I believe initially likely matters more for ROS - a way for the users coming to ROS to continue to use their existing data.
I believe that for this argument to be constructive, first it has to be decided what users are most important, or even likely, to be able to see the priority of FS implementations:
- New users that have no previous data that matters to them.
- Users migrating from Windows, currently using NTFS.
- Users migrating from unix-like operating systems.
Many groups have tried to implement full NTFS functionality over the years. We have managed a point where there is a free NTFS driver that can read NTFS/XP partitions, and can write to them, provided that the following cases are true:
- Nothing "new" is created (folders, files) - Existing files can be written to, but they must be exactly overwritten; you can not truncate or expand the file.
When the system is unmounted, as I understand it, it's marked 'dirty' so that the next time you boot into it, Autochk will take a look at it before booting. Except that Autochk doesn't work on any system that I've worked on, so I had to create a BartPE disc to check the systems occasionally. Personally, I've never seen that driver wipe away a system, but if you were going to start with something, I suppose I'd try to start with that one.
The whole point, however, is that people have tried this and come and gone in the past, and it seems terribly daunting. Is it something that can really be done before a major stable version? If users want to convert their data, can't we build a disc for them that will do that, and if so wouldn't it be smart to move their data to a partition type first, and then install ROS on that partition with the data? It won't necessarily be a one-way-street, as there *are* ways to go back and forth, even if they aren't "pretty".
OTOH, would it be feasible to maybe start with the driver that's been written for Linux? How hard would a Linux driver port to Win32? (Something I wouldn't know; I don't work that low-level with either of them...)
- Mike
Michael B. Trausch wrote:
When the system is unmounted, as I understand it, it's marked 'dirty' so that the next time you boot into it, Autochk will take a look at it before booting. Except that Autochk doesn't work on any system that I've worked on, so I had to create a BartPE disc to check the systems occasionally. Personally, I've never seen that driver wipe away a system, but if you were going to start with something, I suppose I'd try to start with that one.
What do you mean autochk does not work? autochk doesn't do anything for NTFS volumes unless you manually force a check. The entire idea of a transactional filesystem is that it will never need a disk checker to be run on it after the volume is unmounted in a dirty state.
The whole point, however, is that people have tried this and come and gone in the past, and it seems terribly daunting. Is it something that can really be done before a major stable version? If users want to convert their data, can't we build a disc for them that will do that, and if so wouldn't it be smart to move their data to a partition type first, and then install ROS on that partition with the data? It won't necessarily be a one-way-street, as there *are* ways to go back and forth, even if they aren't "pretty".
Converting a partition in place from one format to another isn't really any easier than writing a filesystem driver to access it normally. In fact, it is probably harder. Then the real point is, why would you want to convert to a filesystem that does not work as well?
OTOH, would it be feasible to maybe start with the driver that's been written for Linux? How hard would a Linux driver port to Win32? (Something I wouldn't know; I don't work that low-level with either of them...)
- Mike
Not really. The entire operating environment, upper interfaces, and lower interfaces for a filesystem driver are radically different on Linux than NT.
Steven Edwards wrote:
Windows would have this same problem with NTFS on Linux. If I am under Linux and make changes to files and create files how does linux-NTFS assign the proper SID and such to the files? This has been a long time problem that NTFS has with taking a drive from one Windows box to another. Its the same situation and mostly pointless. 99% of new ReactOS are not going to care and the resources we get from having a better filesystem now can be put in to developing a NTFS replacement.
So? I already said it is a problem translating the other way. The point still is that it is a problem for us to map our views of what a filesystem is and can do onto a unix filesystem, so why bother embracing those problems which at best, have kludgey solutions.
If the ext2fsd supported ext3 journals there would be nothing wrong with using it. I mean hell right now ReactOS does not have a security subsystem so this whole discussion is a moot point. Unless you feel like implementing lsass =)
Thanks Steven
There WOULD be a problem with using it, that is what I have been saying all along. It can't efficiently handle the feature set that ReactOS expects. Maybe you can come up with some way to force it to work, but at this point, I believe that any such solution is a poor one riddled with its own problems, so even if it can be done at all, it is not ideal.
Robert Shearman wrote: You get the same problems when sharing an NTFS filesystem on a dual-boot system or when moving one between computers, unless the SIDs are well-known. I would suggest you check out the Samba presentation at the following URL for information on how Samba4 does NTFS->Unix permissions conversion so you are not stabbing in the dark: http://wiki.winehq.org/WineConf
The problem you are describing has already been solved by them in a slightly-not-kludgey way.
Rob _______________________________________________
That is not the same problem at all. The foreign system does recognize the file structure, the standard file information, alternate data streams, EFS data ( though can not decrypt it obviously ), compressed files, and so on. With regards to the SIDs in the security descriptors, yes, the foreign system does not recognize the unknown SIDs but at least it knows that they are unknown SIDs. On a Linux system when you mount a foreign ext partition, the kernel ends up thinking that it does understand the uids, but it interprets them wrongly by mapping them to random valid users on that system. For example, Joe owns the file on the primary system but when the foreign system mounts the volume, Linux thinks that it is owned by Frank who happens to have been assigned the same uid. NT at least recognizes that it has no idea who Joe is and doesn't try to claim that Frank owns the file.
The problems I am referring to are more like when Linux tries to mount a FAT filesystem. There is no security information on the filesystem that Linux understands, so what does it do? Typically it just pretends that all the files are owned and are accessible to root with r/w/x access. You get the same problem when Linux tries to mount an NTFS volume ( or when ReactOS/NT tries to mount an ext volume ). The information that is actually stored on the disk is not consistent with the information that the OS can deal with, so the information must somehow be faked or translated on the fly, and this causes headaches.
ea wrote:
It seems MS engineers already solved the problem. Try installing the full SFU suite (NIS/AD bridge, NFS server etc.) on a Windows Server 2003 DC and look at a file's properties... _______________________________________________
I am not able to try this myself so can you explain?
I think that something has been lost in this discussion so I would like to restate it. I do not think that supporting ext is bad, I just think that it is not ideal due to the problems involved as a result of the disjoint feature sets. Because it is not ideal, it should not be the filesystem of choice. Because our goal is windows compatibility, NTFS implements all the features that we need quite well, and so it is a much better choice for a preferred filesystem than ext. That is not to say that there isn't something that may be even better than NTFS, only that NTFS is a better choice, all else being equal.
Well, Reiser4 is probably an NT featureset-compatible filesystem made for Linux, but may also work just fine on WinNT. There already exist ACL hacks for both ext2 and reiserfs (v3), which would be what NT typically looks for.
On 5/12/05, Phillip Susi psusi@cfl.rr.com wrote:
That is not the same problem at all. The foreign system does recognize the file structure, the standard file information, alternate data streams, EFS data ( though can not decrypt it obviously ), compressed files, and so on. With regards to the SIDs in the security descriptors, yes, the foreign system does not recognize the unknown SIDs but at least it knows that they are unknown SIDs. On a Linux system when you mount a foreign ext partition, the kernel ends up thinking that it does understand the uids, but it interprets them wrongly by mapping them to random valid users on that system. For example, Joe owns the file on the primary system but when the foreign system mounts the volume, Linux thinks that it is owned by Frank who happens to have been assigned the same uid. NT at least recognizes that it has no idea who Joe is and doesn't try to claim that Frank owns the file.
This is because the ext2 filesystem doesn't contain a username, just the ID. Linux thinks that Frank is the owner of the file, because that ID is the same as the one the filesystem says it is. Windows NT (and many non-NT, including Linux, via ACLs) stores the username in the filesystem, not the ID. This also can fall into a problem where Joe is not the same person on two computers.
Mike Swanson wrote:
Well, Reiser4 is probably an NT featureset-compatible filesystem made for Linux, but may also work just fine on WinNT. There already exist ACL hacks for both ext2 and reiserfs (v3), which would be what NT typically looks for.
I have been reading up on reiser4 and it does look interesting.
This is because the ext2 filesystem doesn't contain a username, just the ID. Linux thinks that Frank is the owner of the file, because that ID is the same as the one the filesystem says it is. Windows NT (and many non-NT, including Linux, via ACLs) stores the username in the filesystem, not the ID. This also can fall into a problem where Joe is not the same person on two computers.
No, NT does NOT store the username in the filesystem. It stores security descriptors which contain SIDs. The mapping of SIDs to human readable names is done by lsass from information either stored in the SAM portion of the registry, or from information obtained from a domain controller or active directory. Because the SIDs are unique, the foreign system will not incorrectly interpret them as referring to someone they are not.
If people want to implement NTFS so badly why not use something like ReiserFS4 for the actual fs, but write a VFS driver to be loaded to act like NTFS. Since ReiserFS supports permissions and things of that nature it should be interesting to see the result. Filesystem encryption could possibly be implemented for files or folders too.
Just a thought.
Does ReiserFS suports fs-level compression? If not, is there any plug-in that allows it to compress files?
João Jerónimo
There has been much talk about a file system that would be compatible with Microsofts implementation of NTFS. There has also been discussion about using something like the ext variants of filesystems, so here are my thoughts:
Option 1 - Write an EXT3 - NTFS VFAT translation driver to allow ext3 to be the underlying filesystem.
I believe Mac OS X uses something like this with the way it can translate Mac OS X:Applications:Utilities into /Applications/Utilities.
Option 2 - Build your own proprietary filesystem incorporating things that people would like in the filesystem.
Option 3 - Build a file system but include features in the operating system that can be turned on and off like Compression or journaling.
Option 4 - Build a filesystem that encrypts itself throught the kernel or a program on the system with a public and a private key, kind of like SSL.
Option 5 - Keep the same file system intact, write a driver that allows users to encrypt a "folder" which is actually a file on the drive, when a user double clicks on a file the folder is either mounted based on a ticketing system or credentials, or, asked for a master password or a password that will give certain but not all priveleges.
Personally this would be my choice.
Tell me what you all think.
Rick Langschultz wrote:
Personally this would be my choice.
Tell me what you all think.
I think that it would make sense to implement whatever filesystems we can -- leaving it to the user to be able to use what they like/trust/want to tinker with.
Here's a question, again, from an uneducated vantage point: Would it be possible and not terribly complicated to incorporate "virtual" IFS drivers that would allow for compression and encryption to be shared among all filesystems, so that they happen in the kernel, and the kernel can support it on filesystems that natively support it as well as those that don't by means of an index file per drive or directory to store the "extended" attributes of a file (compressed, encrypted, whatever)? This would then in theory allow filesystems such as FAT to have transparently compressed and/or encrypted files, even though I would not recommend FAT to and end user to be used in the first place, but that's just my own personal opinion.
- Mike
Actually 'Compression' or the C-bit is one of the file system capability flags an ifs reports to the IO-manager. So in case of NTFS this is an FS-feature But it is also possible to write Filter driver that somehow manages to compress or decompress files (i.e. last part of name is 7z ). The EFS is implemented like that. It lays over NTFS devices to add encryption.
So, BUT: Please, all, tell me, what are you insisting so much on so unimportand features like transparent encryption and transparent compression. In my eyes the first step is to have a FS which is better than FAT and allows ACLs+EA or xattr+posix-acls (NOTE these are mostly isomorphisms). Most people can pass on enc+compr at all. Some use it but they can wait or also pass on it until it can/will be implemented or not implemented. There exist ways to come around that. Even GPL. See 7zip and TrueCrypt. For my requirements TrueCrypt beats NTFS-EFS. So why bother.
I'll now prepare a wikipage which handles an injective (but bijective wanted) translation of acls.
Michael B. Trausch wrote:
Rick Langschultz wrote:
Personally this would be my choice.
Tell me what you all think.
I think that it would make sense to implement whatever filesystems we can -- leaving it to the user to be able to use what they like/trust/want to tinker with.
Here's a question, again, from an uneducated vantage point: Would it be possible and not terribly complicated to incorporate "virtual" IFS drivers that would allow for compression and encryption to be shared among all filesystems, so that they happen in the kernel, and the kernel can support it on filesystems that natively support it as well as those that don't by means of an index file per drive or directory to store the "extended" attributes of a file (compressed, encrypted, whatever)? This would then in theory allow filesystems such as FAT to have transparently compressed and/or encrypted files, even though I would not recommend FAT to and end user to be used in the first place, but that's just my own personal opinion.
- Mike
Ros-dev mailing list Ros-dev@reactos.com http://reactos.com:8080/mailman/listinfo/ros-dev
Robert Köpferl wrote:
Actually 'Compression' or the C-bit is one of the file system capability flags an ifs reports to the IO-manager. So in case of NTFS this is an FS-feature But it is also possible to write Filter driver that somehow manages to compress or decompress files (i.e. last part of name is 7z ). The EFS is implemented like that. It lays over NTFS devices to add encryption.
If the compression is something that is handled by the file system driver itself, then would there be a way to make (most, if not all) of the ReactOS drivers support fs-level compression by making use of one of the better compression libraries (bz2 or lzma comes to mind). In this way, each filesystem driver would only be responsible for how to determine what files are compressed or not, and then passing the compressed files to the operating system kernel to be decompressed transparently before an application accesses it.
So, BUT: Please, all, tell me, what are you insisting so much on so unimportand features like transparent encryption and transparent compression. In my eyes the first step is to have a FS which is better than FAT and allows ACLs+EA or xattr+posix-acls (NOTE these are mostly isomorphisms). Most people can pass on enc+compr at all. Some use it but they can wait or also pass on it until it can/will be implemented or not implemented. There exist ways to come around that. Even GPL. See 7zip and TrueCrypt. For my requirements TrueCrypt beats NTFS-EFS. So why bother.
Transparent encryption and compression are two highly important things, IMHO. It's one of those things that probably would be better off being in the back of someone's mind as they're thinking about how to write the code that will handle the IFS drivers, or the IFS drivers theirselves.
Also, there are some applications that like to silently try to mark it's files both, so in a filesystem that supports it, and even Windows compresses a good bit of %SystemRoot% so that you save space on your hard disk drive.
I think that it might be smart to have ReactOS create and use a GPG key on the user's behalf, solely for the purpose of encrypting their filesystem data when the file owner selects a file to be encrypted on the filesystem. I think that would be smart, and similar to the approach that Windows XP uses in having it's EFS "key" that is encrypted by your Windows XP logon password, be that a local password or your domain password.
Since GPG is fairly strong encryption technology, with fairly speedy results, it seems to me like it is a wise choice. Also, it's generally considered trustworthy by most of those that I have discussed it with.
I'll now prepare a wikipage which handles an injective (but bijective wanted) translation of acls.
Cool. It ought to be interesting. In the end, I agree with you that I'd like to see other, non-FAT file systems supported with good features, such as ACLs and the like, but, since FAT's layout is so simplistic, it probably makes those drivers the easiest ones to create a testbed set for since the filesystem isn't terribly complicated.
Just an idea. Again, I will say up-front, that I have no clue with the code behind ReactOS or the layout of the Win32 API is or would be like in terms of possibility of the ideas mentioned... but if it can be interpreted to be that flexible and yet still be compliant, that could mean a whole new level of security that Windows doesn't (yet) have.
It seems like with some of the rumors going around about Longhorn that they may be doing something similiar, I know someone at one point fairly recently told me that they are trying to get rid of the "must be a Windows administrator to do anything" security problem. Not sure how much actual low-level FS type work would need to really be done with that, I would imagine almost none.
- Mike
Rick Langschultz wrote:
There has been much talk about a file system that would be compatible with Microsofts implementation of NTFS. There has also been discussion about using something like the ext variants of filesystems, so here are my thoughts:
Option 1 - Write an EXT3 - NTFS VFAT translation driver to allow ext3 to be the underlying filesystem.
I believe Mac OS X uses something like this with the way it can translate Mac OS X:Applications:Utilities into /Applications/Utilities.
Keep in mind there is *not* a unique view of system name space: that's how environment subsystems work in NT and in ROS. The client library for the subsystem S will give a process running in the environment S it own view (the "correct one" the process expects) of the system name space. For instance, the Win subsystems makes ("kernel32.dll makes") its processes belive logical volumes have letters attached to them.
A possible macosx.dll would simply translate "Mac OS X:Applications:Utilities" into "\Devices\HarddiskVolume1\Mac OS X\Applications\Utilitites".
Option 2 - Build your own proprietary filesystem incorporating things that people would like in the filesystem.
That's what every file system driver usually does: otherwise only one would exist.
Option 4 - Build a filesystem that encrypts itself throught the kernel or a program on the system with a public and a private key, kind of like SSL.
?
Option 5 - Keep the same file system intact, write a driver that allows users to encrypt a "folder" which is actually a file on the drive, when a user double clicks on a file the folder is either mounted based on a ticketing system or credentials, or, asked for a master password or a password that will give certain but not all priveleges.
That's a filter driver. It's a regular NT IFS technique.
Personally this would be my choice.
Hi,
When building PackageManger under WinXP Rsym crashed with an access violation. Hartex had the same problem and he told me that setting DBG to fixed it for him. And so it did for me.
Someone (I think it was GreateLord) told us to write it to the mailing list.
Maarten Bosma
On 5/10/05, Michael B. Trausch fd0man@gmail.com wrote:
James Tabor wrote: However, on the other side of that, is this: NTFS is just plain *horrible* if you try to gracefully resize it. Very rarely have I heard of an NTFS-resizing operation come out without some form of data loss or filesystem corruption.
I'm sorry to side-tract this conversation, but I must say that I've used ntfsresize many times without fail. I have, however, heard and experienced problems with both Partition Commander and PartitionMagic.
Many people wrote: 1. NTFS is hard/impossible to implement. Undocumented and therefor can/should not be used with ReactOS. 2. ext2/ex3 not good enough for the Windows-NT API, for security, streams, and other support.
I would like to please refute both!
1. NTFS is already fully supported in ReactOS! By Microsoft. As shown by the Captive project. (You did merge back the Captive project Cache manager and file system stuff right?) If ReactOS is to support WinNT device drivers than that means, File-System device drivers as well. Native Knoppix has Captive by default and you can ask my friends of how happy they are for the stuff I saved them from a Bad and crushed XP HD. OK so it is not Implemented yet, but it is 85% and it could be if someone wants to merge Captive subsystems.
2. I have not been to Wineconf-2005, but from the summery what I could see is that one of the key notes was the Samba team. Now ReactOS is going to use Samba right? Or are you going to do your own Windows-for-Workgroups Implementation? Well Samba4 has all these things missing from ext2/ext3, implemented on top, has an extra layer. They call it VNTFS. By use of Samba you will get that for free. Wine is going to use samba to replicate all the needed Win32 API(s). Now in order for ReactOS to use Samba it will need it's POSIX layer pretty polished up.
In summery what I think should be is: A new installation, out of the box, installs ext3 with the Samba VNTFS layers and every thing is happy and free. A user that wants to dual boot or Replace but not loose his/her Windows NTFS HD, will be prompt for the Windows system directory, the NTFS drivers, which are legally his (Hence the existence of the NTFS partition), are copied and happily used by ReactOS. ReactOS does not Create new NTFS partitions but other wise supports them to the letter, by running MS drivers and utilities. It must do that to be true to its goal, as stated on the front page: Run any Windows application and any Windows-NT device driver. Including those from MS, as legally owned by the user. For any other free use of ReactOS the native FS is ext2/3 with Samba VNTFS extensions.
As somewhat of an outsider lurking in the shadows I can tell you guys one thing. ReactOS as a little bit of tendency to the: “Not invented here”. I think that would not compete well with MS's eat all you can, extend and conquer.
Free Life Boaz
Boaz Harrosh schrieb:
- NTFS is already fully supported in ReactOS! By Microsoft. As shown
by the Captive project. (You did merge back the Captive project Cache manager and file system stuff right?) If ReactOS is to support WinNT device drivers than that means, File-System device drivers as well. Native Knoppix has Captive by default and you can ask my friends of how happy they are for the stuff I saved them from a Bad and crushed XP HD. OK so it is not Implemented yet, but it is 85% and it could be if someone wants to merge Captive subsystems.
But this would require that every ReactOS user that wants to use NTFS also has to buy a Windows license. This solution is therefore absolutely useless for us. (I don't know if it is allowed to use this "standard" windows components with operating systems other than Windows)
- I have not been to Wineconf-2005, but from the summery what I could
see is that one of the key notes was the Samba team. Now ReactOS is going to use Samba right? Or are you going to do your own Windows-for-Workgroups Implementation? Well Samba4 has all these things missing from ext2/ext3, implemented on top, has an extra layer. They call it VNTFS. By use of Samba you will get that for free. Wine is going to use samba to replicate all the needed Win32 API(s).
This might be a usable way ... but it's always quite dangerous to emulate file system features. I remember all those problems with OS/2's EADATA stuff on FAT drives ...
Now in order for ReactOS to use Samba it will need it's POSIX layer pretty polished up.
This has nothing to do with the posix subsystem.
In summery what I think should be is: A new installation, out of the box, installs ext3 with the Samba VNTFS layers and every thing is happy and free. A user that wants to dual boot or Replace but not loose his/her Windows NTFS HD, will be prompt for the Windows system directory, the NTFS drivers, which are legally his (Hence the existence of the NTFS partition), are copied and happily used by ReactOS. ReactOS does not Create new NTFS partitions but other wise supports them to the letter, by running MS drivers and utilities. It must do that to be true to its goal, as stated on the front page: Run any Windows application and any Windows-NT device driver. Including those from MS, as legally owned by the user. For any other free use of ReactOS the native FS is ext2/3 with Samba VNTFS extensions.
That's much too complicated. Simply use ReiserFS 4 because it implements all (!) features that are provided by NTFS.
Regards, Mark
Basicly, there are 2 kinds of users. There are those who have windows and and NTFS partition and want to access it from ROS. (perhaps dual boot ROS and win from the one NTFS partition ambie if that is something ROS can be made to do). These people can use ntfs.sys from MS since they have windows.
Then there are those who dont have windows, dont have an existing NTFS partion and dont have a copy of ntfs.sys. Those people can use whatever other solution we come up with (ReactOS implementation of NTFS, some unix FS like ext/reiser etc or whatever else we come up with).
Boaz Harrosh wrote:
- NTFS is already fully supported in ReactOS! By Microsoft. As
shown by
the Captive project. (You did merge back the Captive project Cache manager and file system stuff right?) If ReactOS is to support WinNT device drivers than that means, File-System device drivers as well. Native Knoppix has Captive by default and you can ask my friends of how happy they are for the stuff I saved them from a Bad and crushed XP HD. OK so it is not Implemented yet, but it is 85% and it could be if someone wants to merge Captive subsystems.
Being able to load MS's ntfs driver is a good development goal, as doing so would demonstrate that our kernel really is compatible with NT and provide some means of accessing NTFS volumes from ReactOS with relatively little work. Long term however, we still need our own implementation of a preferred filesystem.
- I have not been to Wineconf-2005, but from the summery what I could
see is that one of the key notes was the Samba team. Now ReactOS is going to use Samba right? Or are you going to do your own Windows-for-Workgroups Implementation? Well Samba4 has all these things missing from ext2/ext3, implemented on top, has an extra layer. They call it VNTFS. By use of Samba you will get that for free. Wine is going to use samba to replicate all the needed Win32 API(s). Now in order for ReactOS to use Samba it will need it's POSIX layer pretty polished up.
No. ReactOS will not use Samba anymore than we use WINE, which is to say, we might borrow bits of code here and there, but as a whole, the project is radically different than what we need. Samba is a server that runs on Linux and emulates a lanman/cifs/whatever MS is calling it these days server. What ReactOS would need is a real implementation of the lanman server on an NT kernel, not a hacked emulation on top of posix.
Heck, I am starting to wonder why we really need to support lanman at all when there are open standards supported by the OSS community already, such as WebDav.
In summery what I think should be is: A new installation, out of the box, installs ext3 with the Samba VNTFS layers and every thing is happy and free. A user that wants to dual boot or Replace but not loose his/her Windows NTFS HD, will be prompt for the Windows system directory, the NTFS drivers, which are legally his (Hence the existence of the NTFS partition), are copied and happily used by ReactOS. ReactOS does not Create new NTFS partitions but other wise supports them to the letter, by running MS drivers and utilities. It must do that to be true to its goal, as stated on the front page: Run any Windows application and any Windows-NT device driver. Including those from MS, as legally owned by the user. For any other free use of ReactOS the native FS is ext2/3 with Samba VNTFS extensions.
As I have said already, I do not believe that ext2/3 adequately supports the features we need, but then I do not completely understand how Samba manages to emulate the things it needs. I will bet though that the emulation does not work as well as having the real thing.
Mark Junker wrote:
That's much too complicated. Simply use ReiserFS 4 because it implements all (!) features that are provided by NTFS.
It does? Maybe I should do some reading on it. Can it really store alternate data streams and NT security descriptors? What about on the fly compression? Encryption? Offline storage?
Phillip Susi schrieb:
Can it really store alternate data streams and NT security descriptors?
Yes, it supports multiple data streams. Just like in NTFS you can store the security information in "invisible" file objects.
What about on the fly compression?
Supported by using the plugin-api.
Encryption?
Supported by using the plugin-api.
Offline storage?
How is this implemented in NTFS? I thought that offline storage is just an ordinary file with a special attribute flag set?
It also supports atomic commits and journaling.
I believe that it's easy to implement all required features using Reiser4 because it differentiates between names and objects. We should be able to use objects without names.
Regards, Mark
-
Boaz Harrosh -
ea -
James Tabor -
Jonathan Wilson -
João Jerónimo Barata de Oliveira -
K McI -
Maarten Bosma -
Mark Junker -
Michael B. Trausch -
Mike Nordell -
Mike Swanson -
Phillip Susi -
reactos -
Rick Langschultz -
Robert Köpferl -
Robert Shearman