Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle: * Automatically queueing asynchronous filesystem operations on a worker thread * Parsing paths and invoking a FS-supplied traversal function for each directory element * Interfacing with the NT cache manager, including flushing data from the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system * Providing a common implementation of byte range locks and oplocks, and disabling fast I/O when they are in use * Caching directory entry lookups * Performing typical input validation/security checks needed by all NT filesystems * Helper tools may also be provided to load pseudo-filesystems (filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include: - Prepare a sketch of the API and callbacks for the IFS helper library (subject to change if necessary) - Implement enough of the wrapper library to implement a simple pseudo-filesystem demonstrating non-cached reads and writes - Add support for cached reads/writes (interaction with the cache manager, caching for on-disk metadata) - Demonstrate a disk-based filesystem without metadata update support (MINIX FS or FAT?) - Demonstrate a disk-based filesystem with full read-write support - If time allows, add support for ancillary features, such as byte locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
[1] - http://www.reactos.org/wiki/Google_Summer_of_Code_2011_Ideas#IFS_Wrapper_Dri...
How about iSCSI?
On Sat, Apr 2, 2011 at 6:26 PM, Bryan Donlan bdonlan@gmail.com wrote:
Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a worker
thread
- Parsing paths and invoking a FS-supplied traversal function for each
directory element
- Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all NT
filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
[1] - http://www.reactos.org/wiki/Google_Summer_of_Code_2011_Ideas#IFS_Wrapper_Dri...
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
On Sat, Apr 2, 2011 at 21:12, WaxDragon waxdragon@gmail.com wrote:
How about iSCSI?
Although I've never used iSCSI myself, as I understand it, it essentially just exposes a SCSI device over the network, right? If so, then handling of iSCSI belongs at the disk device level, not in individual filesystem drivers; the filesystem doesn't need to know that its read/write requests will end up being carried over an Ethernet rather than a SATA cable.
Hi, thanks for a good proposal! Creating such "library" was my dream for quite some time which I never came close to implement due to being busy. It would be great if you could achieve that as part of GSoC.
WBR, Aleksey Bragin.
On Apr 3, 2011, at 2:26 AM, Bryan Donlan wrote:
Hello,
After reading the ReactOS GSoC ideas page, I became interested in the IFS wrapper driver project idea[1], and would appreciate any comments on my project proposal before I formally submit it.
First, a bit about me. I am a student in a dual-major program, studying Computer Science and Japanese at the University of Massachusetts at Amherst. I have had some experience in kernel development previously; as part of an internship at a research project at my University, I wrote some network flow analysis code running in the Linux kernel. While filesystem work will be a first for me, I think it will be an interesting leaning experience. I am fluent in English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern). My freenode username is bd_, and my myReactOS username is bdonlan.
In preparation for writing this proposal, I obtained a copy of the "Windows NT File System Internals: A Developer's Guide" book from O'Reilly. Although a bit dated, it has been enourmously helpful in helping me get an idea of what I'm getting into.
My goal in this project will be to write a library to significantly simplify writing a NT File System Driver. Although writing a kernel-mode filesystem will never be as easy as with, eg, FUSE, it should be possible to abstract away many of the idiosyncracies of the NT filesystem API, at least. For example, the wrapper library may handle:
- Automatically queueing asynchronous filesystem operations on a
worker thread
- Parsing paths and invoking a FS-supplied traversal function for each
directory element
- Interfacing with the NT cache manager, including flushing data from
the cache when a non-cached operation is performed on a cached file, and establishing cache maps for the underlying device for a disk-based file system
- Providing a common implementation of byte range locks and oplocks,
and disabling fast I/O when they are in use
- Caching directory entry lookups
- Performing typical input validation/security checks needed by all NT
filesystems
- Helper tools may also be provided to load pseudo-filesystems
(filesystems with neither network nor disk backing) on an ad-hoc basis
Essentially, the library should make it easy for the filesystem designer to focus on the actual filesystem, rather than the NT IFS interfaces - a read call would be able to skip all the preparation and checks, and directly go to perform the actual I/O.
Major milestones for the project may include:
- Prepare a sketch of the API and callbacks for the IFS helper library
(subject to change if necessary)
- Implement enough of the wrapper library to implement a simple
pseudo-filesystem demonstrating non-cached reads and writes
- Add support for cached reads/writes (interaction with the cache
manager, caching for on-disk metadata)
- Demonstrate a disk-based filesystem without metadata update support
(MINIX FS or FAT?)
- Demonstrate a disk-based filesystem with full read-write support
- If time allows, add support for ancillary features, such as byte
locks, oplocks, and notify watchers.
The interfaces exposed by the filesystem library will of course be clearly documented. Filesystems needing more advanced support will also be able to override any IFS callbacks they choose.
I have an existing consulting commitment that will take some of my time, but I expect to be able to put in 20-30 hours of work per week into this project. Further, I hereby swear that I have not used nor seen the source code to any version of the Windows operating system nor any Microsoft product that may be related to the proposed project that is under a license incompatible with contribution to ReactOS, including but not limited to the leaked Windows 2000 source code and the Windows Research Kernel.
I would appreciate any comments on my proposal before formally submitting it to the GSoC site.
Thanks,
Bryan Donlan
[1] - http://www.reactos.org/wiki/ Google_Summer_of_Code_2011_Ideas#IFS_Wrapper_Driver
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
-
Aleksey Bragin -
Bryan Donlan -
WaxDragon