Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;). It would be most welcome !
Comment Your code scrupulously :)
Best Regards // Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ---------- From: Bryan Donlan bdonlan@gmail.com To: ReactOS Development List ros-dev@reactos.org Date: Sat, 2 Apr 2011 18:26:23 -0400 Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a worker
thread
- Parsing paths and invoking a FS-supplied traversal function for each
directory element
- Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all NT
filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
I would also recommend looking at porting JFS - the sources are available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;). It would be most welcome !
Comment Your code scrupulously :)
Best Regards // Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ---------- From: Bryan Donlan bdonlan@gmail.com To: ReactOS Development List ros-dev@reactos.org Date: Sat, 2 Apr 2011 18:26:23 -0400 Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a worker
thread
- Parsing paths and invoking a FS-supplied traversal function for each
directory element
- Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all NT
filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
"journaling file system in ReactOS" ====> Ext3/4 :D
On Tue, Apr 5, 2011 at 10:02 AM, Adam geekdundee@gmail.com wrote:
I would also recommend looking at porting JFS - the sources are available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;). It would be most welcome !
Comment Your code scrupulously :)
Best Regards // Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ----------
From: Bryan Donlan bdonlan@gmail.com To: ReactOS Development List ros-dev@reactos.org Date: Sat, 2 Apr 2011 18:26:23 -0400 Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a worker
thread
- Parsing paths and invoking a FS-supplied traversal function for each
directory element
- Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all NT
filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
NTFS support:
Dokan->Fuse4WIN->Ntfs-3g
Date: Tue, 5 Apr 2011 11:02:56 +0200 From: elhoir@gmail.com To: ros-dev@reactos.org CC: geekdundee@gmail.com; love.nystrom@gmail.com Subject: Re: [ros-dev] IFS wrapper project proposal draft
"journaling file system in ReactOS" ====> Ext3/4 :D
On Tue, Apr 5, 2011 at 10:02 AM, Adam geekdundee@gmail.com wrote:
I would also recommend looking at porting JFS - the sources are available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been
thinking about
supporting some alternative, OSS, journaling file system in ReactOS, e.g.
ReiserFS,
to allow us to provide the (relative) fail-safety of file transaction
journaling that NTFS has.
Also, hopefully, to enable us to implement NT's access privilege
system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt
e.g ReiserFS to
an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings
of it's own
which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend
"Windows Internals, 4th Edition" for additional/complementary insight.
Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole
book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the
structural
"common sense" to make it a success, and not a mess ;).
It would be most welcome !
Comment Your code scrupulously :)
Best Regards
// Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ----------
From: Bryan Donlan bdonlan@gmail.com
To: ReactOS Development List ros-dev@reactos.org
Date: Sat, 2 Apr 2011 18:26:23 -0400
Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft
Hello,
After reading the ReactOS GSoC ideas page, I became interested in the
IFS wrapper driver project idea[1], and would appreciate any comments
on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program,
studying Computer Science and Japanese at the University of
Massachusetts at Amherst. I have had some experience in kernel
development previously; as part of an internship at a research project
at my University, I wrote some network flow analysis code running in
the Linux kernel. While filesystem work will be a first for me, I
think it will be an interesting leaning experience. I am fluent in
English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern).
My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the
"Windows NT File System Internals: A Developer's Guide" book from
O'Reilly. Although a bit dated, it has been enourmously helpful in
helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly
simplify writing a NT File System Driver. Although writing a
kernel-mode filesystem will never be as easy as with, eg, FUSE, it
should be possible to abstract away many of the idiosyncracies of the
NT filesystem API, at least. For example, the wrapper library may
handle:
* Automatically queueing asynchronous filesystem operations on a worker
thread
* Parsing paths and invoking a FS-supplied traversal function for each
directory element
* Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file,
and establishing cache maps for the underlying device for a disk-based
file system
* Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
* Caching directory entry lookups
* Performing typical input validation/security checks needed by all NT
filesystems
* Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem
designer to focus on the actual filesystem, rather than the NT IFS
interfaces - a read call would be able to skip all the preparation and
checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be
clearly documented. Filesystems needing more advanced support will
also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my
time, but I expect to be able to put in 20-30 hours of work per week
into this project. Further, I hereby swear that I have not used nor
seen the source code to any version of the Windows operating system
nor any Microsoft product that may be related to the proposed project
that is under a license incompatible with contribution to ReactOS,
including but not limited to the leaked Windows 2000 source code and
the Windows Research Kernel.
I would appreciate any comments on my proposal before formally
submitting it to the GSoC site.
Thanks,
Bryan Donlan
Victor,
NTFS should be supported in kernel mode, not via a FUSE driver
On Tue, Apr 5, 2011 at 11:16 AM, victor martinez vicmarcal@hotmail.comwrote:
NTFS support:
Dokan->Fuse4WIN->Ntfs-3g
Date: Tue, 5 Apr 2011 11:02:56 +0200 From: elhoir@gmail.com To: ros-dev@reactos.org CC: geekdundee@gmail.com; love.nystrom@gmail.com Subject: Re: [ros-dev] IFS wrapper project proposal draft
"journaling file system in ReactOS" ====> Ext3/4 :D
On Tue, Apr 5, 2011 at 10:02 AM, Adam geekdundee@gmail.com wrote:
I would also recommend looking at porting JFS - the sources are available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;). It would be most welcome !
Comment Your code scrupulously :)
Best Regards // Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ---------- From: Bryan Donlan bdonlan@gmail.com To: ReactOS Development List ros-dev@reactos.org Date: Sat, 2 Apr 2011 18:26:23 -0400 Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a worker
thread
- Parsing paths and invoking a FS-supplied traversal function for each
directory element
- Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all NT
filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
_______________________________________________ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Why not? :)
Date: Tue, 5 Apr 2011 11:36:30 +0200 From: elhoir@gmail.com To: ros-dev@reactos.org CC: vicmarcal@hotmail.com Subject: Re: [ros-dev] IFS wrapper project proposal draft
Victor,
NTFS should be supported in kernel mode, not via a FUSE driver
On Tue, Apr 5, 2011 at 11:16 AM, victor martinez vicmarcal@hotmail.com wrote:
NTFS support:
Dokan->Fuse4WIN->Ntfs-3g
Date: Tue, 5 Apr 2011 11:02:56 +0200 From: elhoir@gmail.com To: ros-dev@reactos.org
CC: geekdundee@gmail.com; love.nystrom@gmail.com Subject: Re: [ros-dev] IFS wrapper project proposal draft
"journaling file system in ReactOS" ====> Ext3/4 :D
On Tue, Apr 5, 2011 at 10:02 AM, Adam geekdundee@gmail.com wrote:
I would also recommend looking at porting JFS - the sources are available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been
thinking about
supporting some alternative, OSS, journaling file system in ReactOS, e.g.
ReiserFS,
to allow us to provide the (relative) fail-safety of file transaction
journaling that NTFS has.
Also, hopefully, to enable us to implement NT's access privilege
system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt
e.g ReiserFS to
an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings
of it's own
which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend
"Windows Internals, 4th Edition" for additional/complementary insight.
Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole
book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the
structural
"common sense" to make it a success, and not a mess ;).
It would be most welcome !
Comment Your code scrupulously :)
Best Regards
// Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ----------
From: Bryan Donlan bdonlan@gmail.com
To: ReactOS Development List ros-dev@reactos.org
Date: Sat, 2 Apr 2011 18:26:23 -0400
Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft
Hello,
After reading the ReactOS GSoC ideas page, I became interested in the
IFS wrapper driver project idea[1], and would appreciate any comments
on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program,
studying Computer Science and Japanese at the University of
Massachusetts at Amherst. I have had some experience in kernel
development previously; as part of an internship at a research project
at my University, I wrote some network flow analysis code running in
the Linux kernel. While filesystem work will be a first for me, I
think it will be an interesting leaning experience. I am fluent in
English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern).
My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the
"Windows NT File System Internals: A Developer's Guide" book from
O'Reilly. Although a bit dated, it has been enourmously helpful in
helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly
simplify writing a NT File System Driver. Although writing a
kernel-mode filesystem will never be as easy as with, eg, FUSE, it
should be possible to abstract away many of the idiosyncracies of the
NT filesystem API, at least. For example, the wrapper library may
handle:
* Automatically queueing asynchronous filesystem operations on a worker
thread
* Parsing paths and invoking a FS-supplied traversal function for each
directory element
* Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file,
and establishing cache maps for the underlying device for a disk-based
file system
* Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
* Caching directory entry lookups
* Performing typical input validation/security checks needed by all NT
filesystems
* Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem
designer to focus on the actual filesystem, rather than the NT IFS
interfaces - a read call would be able to skip all the preparation and
checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be
clearly documented. Filesystems needing more advanced support will
also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my
time, but I expect to be able to put in 20-30 hours of work per week
into this project. Further, I hereby swear that I have not used nor
seen the source code to any version of the Windows operating system
nor any Microsoft product that may be related to the proposed project
that is under a license incompatible with contribution to ReactOS,
including but not limited to the leaked Windows 2000 source code and
the Windows Research Kernel.
I would appreciate any comments on my proposal before formally
submitting it to the GSoC site.
Thanks,
Bryan Donlan
Because there was a major reason why MS put IFS (NTFS including) in kmode in the first place...
2011/4/5 victor martinez vicmarcal@hotmail.com
Why not? :)
Date: Tue, 5 Apr 2011 11:36:30 +0200
From: elhoir@gmail.com To: ros-dev@reactos.org CC: vicmarcal@hotmail.com
Subject: Re: [ros-dev] IFS wrapper project proposal draft
Victor,
NTFS should be supported in kernel mode, not via a FUSE driver
On Tue, Apr 5, 2011 at 11:16 AM, victor martinez vicmarcal@hotmail.comwrote:
NTFS support:
Dokan->Fuse4WIN->Ntfs-3g
Date: Tue, 5 Apr 2011 11:02:56 +0200 From: elhoir@gmail.com To: ros-dev@reactos.org CC: geekdundee@gmail.com; love.nystrom@gmail.com Subject: Re: [ros-dev] IFS wrapper project proposal draft
"journaling file system in ReactOS" ====> Ext3/4 :D
On Tue, Apr 5, 2011 at 10:02 AM, Adam geekdundee@gmail.com wrote:
I would also recommend looking at porting JFS - the sources are available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;). It would be most welcome !
Comment Your code scrupulously :)
Best Regards // Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ---------- From: Bryan Donlan bdonlan@gmail.com To: ReactOS Development List ros-dev@reactos.org Date: Sat, 2 Apr 2011 18:26:23 -0400 Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a worker
thread
- Parsing paths and invoking a FS-supplied traversal function for each
directory element
- Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all NT
filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
_______________________________________________ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
_______________________________________________ Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
No .. Adam is right about JFS as an option. ext3 has journaling as an add-on after-construction, while JFS is created with journaling from scratch.
Also, JFS originates from OS/2, which is much closer to NT than AIX/Linux. I would surmise that the OSS snapshot is an OS/2 -> AIX -> Linux port.
I'm getting all the relevant sources, and have mailed the maintainer at IBM for a mirror of the original OSS snapshot, since I think that the older the code the closer it is to OS/2, and therefore fewer Linux kernel dependencies to deal with in converting it to an IFS.
WBR // Love
2011/4/5 Javier Agustìn Fernàndez Arroyo elhoir@gmail.com
"journaling file system in ReactOS" ====> Ext3/4 :D
On Tue, Apr 5, 2011 at 10:02 AM, Adam geekdundee@gmail.com wrote:
I would also recommend looking at porting JFS - the sources are available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;). It would be most welcome !
Comment Your code scrupulously :)
Best Regards // Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ----------
From: Bryan Donlan bdonlan@gmail.com To: ReactOS Development List ros-dev@reactos.org Date: Sat, 2 Apr 2011 18:26:23 -0400 Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a worker
thread
- Parsing paths and invoking a FS-supplied traversal function for each
directory element
- Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all NT
filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Hi Adam,
Thanks for the pointer. JFS may be a viable alternative to NTFS indeed, they probably have the same root (HPFS)?
The bulk of porting it to ReactOS will probably be "un-porting" it's Linux kernel dependency patches. I'm fetching the whole CVS repository from SourceForge right now, but I'm not sure when I will get the time to start eyeballing it .. I should be doing RL stuff today but find my butt glued to the chair again (Water_stoppage==Shower_impossible==I_refuse_to_go_public)!
Peace // Love
On Tue, Apr 5, 2011 at 3:02 PM, Adam geekdundee@gmail.com wrote:
I would also recommend looking at porting JFS - the sources are available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;). It would be most welcome !
Comment Your code scrupulously :)
Best Regards // Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ----------
From: Bryan Donlan bdonlan@gmail.com To: ReactOS Development List ros-dev@reactos.org Date: Sat, 2 Apr 2011 18:26:23 -0400 Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a worker
thread
- Parsing paths and invoking a FS-supplied traversal function for each
directory element
- Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all NT
filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
AFAIK only NTFS has this root; JFS has been around for a while in the AIX systems. It was ported to OS/2 Warp 4.5 later on.
Well at least they're both better than all that ext1/ext2/ext3/ext4/extπ/whatever gibberish. The biggest problem is (and it has actually happened to me before) these guys can run out of "inodes" and what not.
On Tue, 05 Apr 2011 19:40:09 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Adam,
Thanks for the pointer. JFS may be a viable alternative to NTFS indeed, they probably have the same root (HPFS)?
The bulk of porting it to ReactOS will probably be "un-porting" it's Linux kernel dependency patches. I'm fetching the whole CVS repository from SourceForge right now, but I'm not sure when I will get the time to start eyeballing it .. I should be doing RL stuff today but find my butt glued to the chair again (Water_stoppage==Shower_impossible==I_refuse_to_go_public)!
Peace // Love
On Tue, Apr 5, 2011 at 3:02 PM, Adam geekdundee@gmail.com wrote:
I would also recommend looking at porting JFS - the sources are available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;). It would be most welcome !
Comment Your code scrupulously :)
Best Regards // Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ----------
From: Bryan Donlan bdonlan@gmail.com To: ReactOS Development List ros-dev@reactos.org Date: Sat, 2 Apr 2011 18:26:23 -0400 Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a
worker thread
- Parsing paths and invoking a FS-supplied traversal function for each
directory element
- Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all NT
filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
You sure it was ported AIX -> OS/2, and not the other way around?
WBR // Love
On Tue, Apr 5, 2011 at 5:57 PM, Adam geekdundee@gmail.com wrote:
AFAIK only NTFS has this root; JFS has been around for a while in the AIX systems. It was ported to OS/2 Warp 4.5 later on.
Well at least they're both better than all that ext1/ext2/ext3/ext4/extπ/whatever gibberish. The biggest problem is (and it has actually happened to me before) these guys can run out of "inodes" and what not.
On Tue, 05 Apr 2011 19:40:09 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Adam,
Thanks for the pointer. JFS may be a viable alternative to NTFS indeed, they probably have the same root (HPFS)?
The bulk of porting it to ReactOS will probably be "un-porting" it's Linux kernel dependency patches. I'm fetching the whole CVS repository from SourceForge right now, but I'm not sure when I will get the time to start eyeballing it .. I should be doing RL stuff today but find my butt glued to the chair again (Water_stoppage==Shower_impossible==I_refuse_to_go_public)!
Peace // Love
On Tue, Apr 5, 2011 at 3:02 PM, Adam geekdundee@gmail.com wrote:
I would also recommend looking at porting JFS - the sources are
available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom <love.nystrom@gmail.com
wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;). It would be most welcome !
Comment Your code scrupulously :)
Best Regards // Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ----------
From: Bryan Donlan bdonlan@gmail.com To: ReactOS Development List ros-dev@reactos.org Date: Sat, 2 Apr 2011 18:26:23 -0400 Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a worker
thread
- Parsing paths and invoking a FS-supplied traversal function for each
directory element
- Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all NT
filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
That's what I remember. I think it was originally JFS1 or something and it became JFS later on.
It was then made for OS/2 Warp and then it got open-sourced for Linux distros.
I could be wrong; but I am 98% sure I am correct about this.
On Tue, 05 Apr 2011 22:15:50 +1000, Love Nystrom love.nystrom@gmail.com wrote:
You sure it was ported AIX -> OS/2, and not the other way around?
WBR // Love
On Tue, Apr 5, 2011 at 5:57 PM, Adam geekdundee@gmail.com wrote:
AFAIK only NTFS has this root; JFS has been around for a while in the AIX systems. It was ported to OS/2 Warp 4.5 later on.
Well at least they're both better than all that ext1/ext2/ext3/ext4/extπ/whatever gibberish. The biggest problem is (and it has actually happened to me before) these guys can run out of "inodes" and what not.
On Tue, 05 Apr 2011 19:40:09 +1000, Love Nystrom love.nystrom@gmail.com wrote:
Hi Adam,
Thanks for the pointer. JFS may be a viable alternative to NTFS indeed, they probably have the same root (HPFS)?
The bulk of porting it to ReactOS will probably be "un-porting" it's Linux kernel dependency patches. I'm fetching the whole CVS repository from SourceForge right now, but I'm not sure when I will get the time to start eyeballing it .. I should be doing RL stuff today but find my butt glued to the chair again (Water_stoppage==Shower_impossible==I_refuse_to_go_public)!
Peace // Love
On Tue, Apr 5, 2011 at 3:02 PM, Adam geekdundee@gmail.com wrote:
I would also recommend looking at porting JFS - the sources are
available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom <love.nystrom@gmail.com
wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;). It would be most welcome !
Comment Your code scrupulously :)
Best Regards // Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ----------
From: Bryan Donlan bdonlan@gmail.com To: ReactOS Development List ros-dev@reactos.org Date: Sat, 2 Apr 2011 18:26:23 -0400 Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a
worker thread
- Parsing paths and invoking a FS-supplied traversal function for
each directory element
- Interfacing with the NT cache manager, including flushing data
from the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all
NT filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper
library (subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update
support (MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
Aha .. I understand. Still, starting with the oldest possible version might be the way to go to avoid being overwhelmed by Linux kernel dependencies.
But I may be wrong .. it'll get clearer when I can get the time to eyeball the code.
WBR // Love
On Tue, Apr 5, 2011 at 7:17 PM, Adam geekdundee@gmail.com wrote:
That's what I remember. I think it was originally JFS1 or something and it became JFS later on.
It was then made for OS/2 Warp and then it got open-sourced for Linux distros.
I could be wrong; but I am 98% sure I am correct about this.
On Tue, 05 Apr 2011 22:15:50 +1000, Love Nystrom love.nystrom@gmail.com wrote:
You sure it was ported AIX -> OS/2, and not the other way around?
WBR // Love
On Tue, Apr 5, 2011 at 5:57 PM, Adam geekdundee@gmail.com wrote:
AFAIK only NTFS has this root; JFS has been around for a while in the AIX
systems. It was ported to OS/2 Warp 4.5 later on.
Well at least they're both better than all that ext1/ext2/ext3/ext4/extπ/whatever gibberish. The biggest problem is (and it has actually happened to me before) these guys can run out of "inodes" and what not.
On Tue, 05 Apr 2011 19:40:09 +1000, Love Nystrom <love.nystrom@gmail.com
wrote:
Hi Adam,
Thanks for the pointer. JFS may be a viable alternative to NTFS indeed, they probably have the same root (HPFS)?
The bulk of porting it to ReactOS will probably be "un-porting" it's Linux kernel dependency patches. I'm fetching the whole CVS repository from SourceForge right now, but I'm not sure when I will get the time to start eyeballing it .. I should be doing RL stuff today but find my butt glued to the chair again (Water_stoppage==Shower_impossible==I_refuse_to_go_public)!
Peace // Love
On Tue, Apr 5, 2011 at 3:02 PM, Adam geekdundee@gmail.com wrote:
I would also recommend looking at porting JFS - the sources are
available. This is the file system that IBM originally used in OS/2 Warp 4.5X (and also AIX if I am not mistaken) and is a very solid file system. Something IBM seemed to be doing right.
I am not sure about all the details of it, but AFAIK it also supports things like ACLs too.
On Tue, 05 Apr 2011 17:56:40 +1000, Love Nystrom < love.nystrom@gmail.com
wrote:
Hi Bryan,
Just like Aleksey, I would welcome an effort to simplify IFS implementation.
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files.
So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think).
It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;). It would be most welcome !
Comment Your code scrupulously :)
Best Regards // Love
On Sun, Apr 3, 2011 at 1:55 PM, ros-dev-request@reactos.org wrote:
---------- Forwarded message ----------
From: Bryan Donlan bdonlan@gmail.com > To: ReactOS Development List ros-dev@reactos.org > Date: Sat, 2 Apr 2011 18:26:23 -0400 > Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft > Hello, > > After reading the ReactOS GSoC ideas page, I became interested in the > IFS wrapper driver project idea[1], and would appreciate any comments > on my project proposal before I formally submit it. > > First, a bit about me. I am a student in a dual-major program, > studying Computer Science and Japanese at the University of > Massachusetts at Amherst. I have had some experience in kernel > development previously; as part of an internship at a research > project > at my University, I wrote some network flow analysis code running in > the Linux kernel. While filesystem work will be a first for me, I > think it will be an interesting leaning experience. I am fluent in > English and Japanese, and am in the UTC-0400 timezone > (TZ=US/Eastern). > My freenode username is bd_, and my myReactOS username is bdonlan. > > In preparation for writing this proposal, I obtained a copy of the > "Windows NT File System Internals: A Developer's Guide" book from > O'Reilly. Although a bit dated, it has been enourmously helpful in > helping me get an idea of what I'm getting into. > > My goal in this project will be to write a library to significantly > simplify writing a NT File System Driver. Although writing a > kernel-mode filesystem will never be as easy as with, eg, FUSE, it > should be possible to abstract away many of the idiosyncracies of the > NT filesystem API, at least. For example, the wrapper library may > handle: > * Automatically queueing asynchronous filesystem operations on a > worker > thread > * Parsing paths and invoking a FS-supplied traversal function for > each > directory element > * Interfacing with the NT cache manager, including flushing data from > the cache when a non-cached operation is performed on a cached file, > and establishing cache maps for the underlying device for a > disk-based > file system > * Providing a common implementation of byte range locks and oplocks, > and disabling fast I/O when they are in use > * Caching directory entry lookups > * Performing typical input validation/security checks needed by all > NT > filesystems > * Helper tools may also be provided to load pseudo-filesystems > (filesystems with neither network nor disk backing) on an ad-hoc > basis > > Essentially, the library should make it easy for the filesystem > designer to focus on the actual filesystem, rather than the NT IFS > interfaces - a read call would be able to skip all the preparation > and > checks, and directly go to perform the actual I/O. > > Major milestones for the project may include: > - Prepare a sketch of the API and callbacks for the IFS helper > library > (subject to change if necessary) > - Implement enough of the wrapper library to implement a simple > pseudo-filesystem demonstrating non-cached reads and writes > - Add support for cached reads/writes (interaction with the cache > manager, caching for on-disk metadata) > - Demonstrate a disk-based filesystem without metadata update support > (MINIX FS or FAT?) > - Demonstrate a disk-based filesystem with full read-write support > - If time allows, add support for ancillary features, such as byte > locks, oplocks, and notify watchers. > > The interfaces exposed by the filesystem library will of course be > clearly documented. Filesystems needing more advanced support will > also be able to override any IFS callbacks they choose. > > I have an existing consulting commitment that will take some of my > time, but I expect to be able to put in 20-30 hours of work per week > into this project. Further, I hereby swear that I have not used nor > seen the source code to any version of the Windows operating system > nor any Microsoft product that may be related to the proposed project > that is under a license incompatible with contribution to ReactOS, > including but not limited to the leaked Windows 2000 source code and > the Windows Research Kernel. > > I would appreciate any comments on my proposal before formally > submitting it to the GSoC site. > > Thanks, > > Bryan Donlan > > > > > --
Using Opera's revolutionary email client: http://www.opera.com/mail/
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
-- Using Opera's revolutionary email client: http://www.opera.com/mail/
On Tue, Apr 5, 2011 at 03:56, Love Nystrom love.nystrom@gmail.com wrote:
Hi Bryan, Just like Aleksey, I would welcome an effort to simplify IFS implementation. Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file system in ReactOS, e.g. ReiserFS, to allow us to provide the (relative) fail-safety of file transaction journaling that NTFS has. Also, hopefully, to enable us to implement NT's access privilege system (Security) on files. So far, it remains just a thought, since I can't allocate the time to adapt e.g ReiserFS to an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings of it's own which You may need to address as well in implementing Your proposal.
A nice idea, but I think I'll have my hands full enough trying to refactor the IFS API, without having to figure out the platform dependencies in an existing filesystem (such as JFS or ReiserFS or even ntfs-3g) at the same time... :) If there's time at the end I suppose it might be an interesting thing to try, or if someone else wants to work on the filesystem porting side I could work with them on it.
I do hope to create an API that is similar to what unix VFSes expose, so it ought to make porting easier in the end.
In addition to "Windows NT File System Internals" I would also recommend "Windows Internals, 4th Edition" for additional/complementary insight. Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole book is strongly recommended (by everyone, I think). It will be a BIG job I think (thesis big), and I hope You have the structural "common sense" to make it a success, and not a mess ;).
Only one way to find out!
On Wed, Apr 6, 2011 at 2:52 AM, Bryan Donlan bdonlan@gmail.com wrote:
On Tue, Apr 5, 2011 at 03:56, Love Nystrom love.nystrom@gmail.com wrote:
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file
system in ReactOS,
e.g. ReiserFS,
A nice idea, but I think I'll have my hands full enough trying to refactor the IFS API, without having to figure out the platform dependencies in an existing filesystem (such as JFS or ReiserFS or even ntfs-3g) at the same time... :) If there's time at the end I suppose it might be an interesting thing to try, or if someone else wants to work on the filesystem porting side I could work with them on it.
Well .. I didn't actually mean for *You* to port ReiserFS, as well as refactoring IFS ;)
In fact, there are some OSS ReiserFS IFS implementations already available. A good summary of the situation is here: http://www.reactos.org/wiki/File_Systems/ReiserFS Also some forum points of view: http://www.reactos.org/forum/viewtopic.php?t=1689
ResierFS 3 has two OSS IFS implementations, one at SourceForge and one at Umeå University, but according to Hans Reiser himself, version 3 is considered deprecated due to design shortcomings. ReiserFS 4 has a Student Project in Switzerland, which they are willing to release to ReactOS.
AFAIK all of these implementations are built with MS IFS SDK, so they will require some work to build properly under RosBE, and testing testing testing. Unfortunately I myself am bogged down by RL, so I can't find the time, at least not now .. maybe next winter :(
I do hope to create an API that is similar to what unix VFSes expose, so it ought to make porting easier in the end.
That might make it easier to port e.g JFS to ReactOS.
After contacting Dave Kleinkamp, maintainer of the Linux JFS implementation, which lives at SourceForge, I have retrieved the original IBM OSS snapshot, JFS 0.0.1, available by FTP from 147.52.159.12/system/
According to him, the Linux version was ported from OS/2, as I suspected, and not from AIX as we previously concluded, so the 0.0.1 code should be at least a good reference, if not a starting point, for an IFS port. The current Linux JFS supports both extended attributes and ACLs, making it attractive as a starting point, possibly eased by Your wrapper, but all of these implementations are tied pretty hard to the Unix file model, AFAI could see, so it may take a *lot* of work to create an IFS port.
"common sense" to make it a success, and not a mess ;). Only one way to find out!
Oh so true ;)
Good Luck and Best Regards // Love N
Let us not limit ourselves to raw JFS though. Why not improve it while we can? Add support for on-the-fly file encryption in there for example. Just thinkin. :)
On Sat, 09 Apr 2011 20:41:24 +1000, Love Nystrom love.nystrom@gmail.com wrote:
On Wed, Apr 6, 2011 at 2:52 AM, Bryan Donlan bdonlan@gmail.com wrote:
On Tue, Apr 5, 2011 at 03:56, Love Nystrom love.nystrom@gmail.com wrote:
Seen as NTFS is (and always will be?) not well documented, I've been thinking about supporting some alternative, OSS, journaling file
system in ReactOS,
e.g. ReiserFS,
A nice idea, but I think I'll have my hands full enough trying to refactor the IFS API, without having to figure out the platform dependencies in an existing filesystem (such as JFS or ReiserFS or even ntfs-3g) at the same time... :) If there's time at the end I suppose it might be an interesting thing to try, or if someone else wants to work on the filesystem porting side I could work with them on it.
Well .. I didn't actually mean for *You* to port ReiserFS, as well as refactoring IFS ;)
In fact, there are some OSS ReiserFS IFS implementations already available. A good summary of the situation is here: http://www.reactos.org/wiki/File_Systems/ReiserFS Also some forum points of view: http://www.reactos.org/forum/viewtopic.php?t=1689
ResierFS 3 has two OSS IFS implementations, one at SourceForge and one at Umeå University, but according to Hans Reiser himself, version 3 is considered deprecated due to design shortcomings. ReiserFS 4 has a Student Project in Switzerland, which they are willing to release to ReactOS.
AFAIK all of these implementations are built with MS IFS SDK, so they will require some work to build properly under RosBE, and testing testing testing. Unfortunately I myself am bogged down by RL, so I can't find the time, at least not now .. maybe next winter :(
I do hope to create an API that is similar to what unix VFSes expose, so it ought to make porting easier in the end.
That might make it easier to port e.g JFS to ReactOS.
After contacting Dave Kleinkamp, maintainer of the Linux JFS implementation, which lives at SourceForge, I have retrieved the original IBM OSS snapshot, JFS 0.0.1, available by FTP from 147.52.159.12/system/
According to him, the Linux version was ported from OS/2, as I suspected, and not from AIX as we previously concluded, so the 0.0.1 code should be at least a good reference, if not a starting point, for an IFS port. The current Linux JFS supports both extended attributes and ACLs, making it attractive as a starting point, possibly eased by Your wrapper, but all of these implementations are tied pretty hard to the Unix file model, AFAI could see, so it may take a *lot* of work to create an IFS port.
"common sense" to make it a success, and not a mess ;). Only one way to find out!
Oh so true ;)
Good Luck and Best Regards // Love N
-
Adam -
Bryan Donlan -
Javier Agustìn Fernàndez Arroyo -
Love Nystrom -
Olaf Siejka -
victor martinez