[arty] 49423: [CACHE] The cache manager rewrite I started years ago has finally appeared in ReactOS' trunk and although at this point it's not quite perfectly integrated, it's enough to boot up the...

Author: arty
Date: Tue Nov  2 02:32:39 2010
New Revision: 49423

URL: http://svn.reactos.org/svn/reactos?rev=49423&view=rev
Log:
[CACHE]

The cache manager rewrite I started years ago has finally appeared in
ReactOS' trunk and although at this point it's not quite perfectly
integrated, it's enough to boot up the bootcd or livecd. To check out
the more mature original, check out arty-newcc-reactos, branch
arty-newcc on bitbucket.org . Amine Khaldi encouraged me quite a bit
to not give up on it, and was able to reach out and be an advocate
when i really wasn't able to.  Others agree that the time has come to
begin removing the old cache manager. I expect the remaining problems
in the version going to trunk will be taken care of relatively
quickly.

The motivation for this effort lies in the particularly hairy
relationship between ReactOS' cache manager and data sections. This
code completely removes page sharing between cache manager and section
and reimagines cache manager as being a facility layered on the memory
manager, not really caring about individual pages, but simply managing
data section objects where caching might occur.

It took me about 2 years to do the first pass of this rewrite and most
of this year to fix some lingering issues, properly implement demand
paging in ReactOS (code which didn't come with this patch in a
recognizable form), and finish getting the PrivateCacheMap and
SharedCacheMap relationship correct.

Currently, the new ntoskrnl/cache directory contains an own
implementation of data file sections. After things have settled down,
we can begin to deprecate and remove the parts of ReactOS' section
implementation that depend on a close relationship with cache
manager. Eventually, I think that the extra code added to
ntoskrnl/cache/section will be removed and ReactOS' own sections will
replace the use of the special MM_CACHE_SECTION_SEGMENT in the cache
path.

Note also, that this makes all cache manager (and new section parts)
use wide file offsets. If my section code were to take over other
parts of the ReactOS memory manager, they would also benefit from
these improvements.

I invite anyone who wants to to peek at this code and fix whatever
bugs can be found.

Added:
    trunk/reactos/include/ndk/inline_ntcurrentteb.h   (with props)
    trunk/reactos/ntoskrnl/cache/
    trunk/reactos/ntoskrnl/cache/cachesub.c   (with props)
    trunk/reactos/ntoskrnl/cache/copysup.c   (with props)
    trunk/reactos/ntoskrnl/cache/fssup.c   (with props)
    trunk/reactos/ntoskrnl/cache/lazyrite.c   (with props)
    trunk/reactos/ntoskrnl/cache/logsup.c   (with props)
    trunk/reactos/ntoskrnl/cache/mdlsup.c   (with props)
    trunk/reactos/ntoskrnl/cache/newcc.h   (with props)
    trunk/reactos/ntoskrnl/cache/pinsup.c   (with props)
    trunk/reactos/ntoskrnl/cache/section/
    trunk/reactos/ntoskrnl/cache/section/data.c   (with props)
    trunk/reactos/ntoskrnl/cache/section/fault.c   (with props)
    trunk/reactos/ntoskrnl/cache/section/io.c   (with props)
    trunk/reactos/ntoskrnl/cache/section/newmm.h   (with props)
    trunk/reactos/ntoskrnl/cache/section/reqtools.c   (with props)
    trunk/reactos/ntoskrnl/cache/section/sptab.c   (with props)
    trunk/reactos/ntoskrnl/cache/section/swapout.c   (with props)
Modified:
    trunk/reactos/config.template.rbuild
    trunk/reactos/include/ndk/psfuncs.h
    trunk/reactos/include/psdk/winnt.h
    trunk/reactos/ntoskrnl/include/internal/mm.h
    trunk/reactos/ntoskrnl/include/internal/ps.h
    trunk/reactos/ntoskrnl/mm/i386/page.c
    trunk/reactos/ntoskrnl/mm/mmfault.c
    trunk/reactos/ntoskrnl/mm/mminit.c
    trunk/reactos/ntoskrnl/mm/rmap.c
    trunk/reactos/ntoskrnl/mm/section.c
    trunk/reactos/ntoskrnl/ntoskrnl-generic.rbuild
    trunk/reactos/ntoskrnl/po/poshtdwn.c
    trunk/reactos/ntoskrnl/po/power.c
    trunk/reactos/ntoskrnl/ps/kill.c

[This mail would be too long, it was shortened to contain the URLs only.]

Modified: trunk/reactos/config.template.rbuild
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/config.template.rbuild?rev=...

Added: trunk/reactos/include/ndk/inline_ntcurrentteb.h
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/include/ndk/inline_ntcurren...

Modified: trunk/reactos/include/ndk/psfuncs.h
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/include/ndk/psfuncs.h?rev=4...

Modified: trunk/reactos/include/psdk/winnt.h
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/include/psdk/winnt.h?rev=49...

Added: trunk/reactos/ntoskrnl/cache/cachesub.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/cachesub.c?r...

Added: trunk/reactos/ntoskrnl/cache/copysup.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/copysup.c?re...

Added: trunk/reactos/ntoskrnl/cache/fssup.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/fssup.c?rev=...

Added: trunk/reactos/ntoskrnl/cache/lazyrite.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/lazyrite.c?r...

Added: trunk/reactos/ntoskrnl/cache/logsup.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/logsup.c?rev...

Added: trunk/reactos/ntoskrnl/cache/mdlsup.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/mdlsup.c?rev...

Added: trunk/reactos/ntoskrnl/cache/newcc.h
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/newcc.h?rev=...

Added: trunk/reactos/ntoskrnl/cache/pinsup.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/pinsup.c?rev...

Added: trunk/reactos/ntoskrnl/cache/section/data.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/section/data...

Added: trunk/reactos/ntoskrnl/cache/section/fault.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/section/faul...

Added: trunk/reactos/ntoskrnl/cache/section/io.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/section/io.c...

Added: trunk/reactos/ntoskrnl/cache/section/newmm.h
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/section/newm...

Added: trunk/reactos/ntoskrnl/cache/section/reqtools.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/section/reqt...

Added: trunk/reactos/ntoskrnl/cache/section/sptab.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/section/spta...

Added: trunk/reactos/ntoskrnl/cache/section/swapout.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/cache/section/swap...

Modified: trunk/reactos/ntoskrnl/include/internal/mm.h
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/include/internal/m...

Modified: trunk/reactos/ntoskrnl/include/internal/ps.h
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/include/internal/p...

Modified: trunk/reactos/ntoskrnl/mm/i386/page.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/mm/i386/page.c?rev...

Modified: trunk/reactos/ntoskrnl/mm/mmfault.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/mm/mmfault.c?rev=4...

Modified: trunk/reactos/ntoskrnl/mm/mminit.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/mm/mminit.c?rev=49...

Modified: trunk/reactos/ntoskrnl/mm/rmap.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/mm/rmap.c?rev=4942...

Modified: trunk/reactos/ntoskrnl/mm/section.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/mm/section.c?rev=4...

Modified: trunk/reactos/ntoskrnl/ntoskrnl-generic.rbuild
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/ntoskrnl-generic.r...

Modified: trunk/reactos/ntoskrnl/po/poshtdwn.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/po/poshtdwn.c?rev=...

Modified: trunk/reactos/ntoskrnl/po/power.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/po/power.c?rev=494...

Modified: trunk/reactos/ntoskrnl/ps/kill.c
URL: http://svn.reactos.org/svn/reactos/trunk/reactos/ntoskrnl/ps/kill.c?rev=4942...