New subject: IFS wrapper project proposal draft

5 Apr 2011

Hi Bryan,

Just like Aleksey, I would welcome an effort to simplify IFS implementation.

Seen as NTFS is (and always will be?) not well documented, I've been
thinking about
supporting some alternative, OSS, journaling file system in ReactOS, e.g.
ReiserFS,
to allow us to provide the (relative) fail-safety of file transaction
journaling that NTFS has.
Also, hopefully, to enable us to implement NT's access privilege
system (Security) on files.

So far, it remains just a thought, since I can't allocate the time to adapt
e.g ReiserFS to
an NT IFS, and if I'm not mistaken, our IFS implementation has shortcomings
of it's own
which You may need to address as well in implementing Your proposal.

In addition to "Windows NT File System Internals" I would also recommend
"Windows Internals, 4th Edition" for additional/complementary insight.
Especially Ch.12 "File Systems", but Ch.8-12 are all relevant, and the whole
book is strongly recommended (by everyone, I think).

It will be a BIG job I think (thesis big), and I hope You have the
structural
"common sense" to make it a success, and not a mess ;).
It would be most welcome !

Comment Your code scrupulously :)

Best Regards
// Love

On Sun, Apr 3, 2011 at 1:55 PM, &lt;ros-dev-request(a)reactos.org&gt; wrote:

...
  ---------- Forwarded message ----------
 From: Bryan Donlan &lt;bdonlan(a)gmail.com&gt;
 To: ReactOS Development List &lt;ros-dev(a)reactos.org&gt;
 Date: Sat, 2 Apr 2011 18:26:23 -0400
 Subject: [ros-dev] [GSoC] IFS wrapper project proposal draft
 Hello,

 After reading the ReactOS GSoC ideas page, I became interested in the
 IFS wrapper driver project idea[1], and would appreciate any comments
 on my project proposal before I formally submit it.

 First, a bit about me. I am a student in a dual-major program,
 studying Computer Science and Japanese at the University of
 Massachusetts at Amherst. I have had some experience in kernel
 development previously; as part of an internship at a research project
 at my University, I wrote some network flow analysis code running in
 the Linux kernel. While filesystem work will be a first for me, I
 think it will be an interesting leaning experience. I am fluent in
 English and Japanese, and am in the UTC-0400 timezone (TZ=US/Eastern).
 My freenode username is bd_, and my myReactOS username is bdonlan.

 In preparation for writing this proposal, I obtained a copy of the
 "Windows NT File System Internals: A Developer's Guide" book from
 O'Reilly. Although a bit dated, it has been enourmously helpful in
 helping me get an idea of what I'm getting into.

 My goal in this project will be to write a library to significantly
 simplify writing a NT File System Driver. Although writing a
 kernel-mode filesystem will never be as easy as with, eg, FUSE, it
 should be possible to abstract away many of the idiosyncracies of the
 NT filesystem API, at least. For example, the wrapper library may
 handle:
 * Automatically queueing asynchronous filesystem operations on a worker
 thread
 * Parsing paths and invoking a FS-supplied traversal function for each
 directory element
 * Interfacing with the NT cache manager, including flushing data from
 the cache when a non-cached operation is performed on a cached file,
 and establishing cache maps for the underlying device for a disk-based
 file system
 * Providing a common implementation of byte range locks and oplocks,
 and disabling fast I/O when they are in use
 * Caching directory entry lookups
 * Performing typical input validation/security checks needed by all NT
 filesystems
 * Helper tools may also be provided to load pseudo-filesystems
 (filesystems with neither network nor disk backing) on an ad-hoc basis

 Essentially, the library should make it easy for the filesystem
 designer to focus on the actual filesystem, rather than the NT IFS
 interfaces - a read call would be able to skip all the preparation and
 checks, and directly go to perform the actual I/O.

 Major milestones for the project may include:
 - Prepare a sketch of the API and callbacks for the IFS helper library
 (subject to change if necessary)
 - Implement enough of the wrapper library to implement a simple
 pseudo-filesystem demonstrating non-cached reads and writes
 - Add support for cached reads/writes (interaction with the cache
 manager, caching for on-disk metadata)
 - Demonstrate a disk-based filesystem without metadata update support
 (MINIX FS or FAT?)
 - Demonstrate a disk-based filesystem with full read-write support
 - If time allows, add support for ancillary features, such as byte
 locks, oplocks, and notify watchers.

 The interfaces exposed by the filesystem library will of course be
 clearly documented. Filesystems needing more advanced support will
 also be able to override any IFS callbacks they choose.

 I have an existing consulting commitment that will take some of my
 time, but I expect to be able to put in 20-30 hours of work per week
 into this project. Further, I hereby swear that I have not used nor
 seen the source code to any version of the Windows operating system
 nor any Microsoft product that may be related to the proposed project
 that is under a license incompatible with contribution to ReactOS,
 including but not limited to the leaked Windows 2000 source code and
 the Windows Research Kernel.

 I would appreciate any comments on my proposal before formally
 submitting it to the GSoC site.

 Thanks,

 Bryan Donlan

Re: [ros-dev] IFS wrapper project proposal draft