[arty] 44847: Reorganization and rewrite of the paging code, along with stray changes from trunk. Not totally bug free yet, but mature enough to show. Primary change is seperate concerns in fault handling type code (including page out), to isolate each component's fault handlers from the generic fault code, as well as to allow us to move resource acquistion out of the minefield of locks and deadlocks in the handlers. We *never* break a lock we receive locked anymore; if we need to acquire resour

Author: arty
Date: Fri Jan  1 11:22:47 2010
New Revision: 44847

URL: http://svn.reactos.org/svn/reactos?rev=44847&view=rev
Log:
Reorganization and rewrite of the paging code, along with stray changes from trunk.
Not totally bug free yet, but mature enough to show.

Primary change is seperate concerns in fault handling type code (including page out),
to isolate each component's fault handlers from the generic fault code, as well as to
allow us to move resource acquistion out of the minefield of locks and deadlocks in
the handlers.  We *never* break a lock we receive locked anymore; if we need to acquire
resources that would deadlock one of the locks we hold, we punt by filling in the 
appropriate fields of MM_REQUIRED_RESORUCE and returning STATUS_MORE_PROCESSING_REQUIRED.
Good side effects are hygiene and atomicity in the handlers; each handler's code can think
of itself as atomic with respect to others operating on the same address space or segment.
This leads to less duplication of code but a slight increase in redundant computation that
we can eliminate later.  Special support has been added for paging out shared section
pages, but it doesn't feel terrible.

In fault handlers, the following additional return values are available:

STATUS_MORE_PROCESSING_REQUIRED -> call the DoAcquisition function from the
MM_REQUIRED_RESOURCES and recall the fault handler.
STATUS_SUCCESS + 1 -> wait on the paging wait event.  we got a sentinel page entry,
indicating that somebody else is operating on the slot we want.

Introduce a sentinel value of MM_WAIT_ENTRY to mean that somebody else is operating on a
page slot in either a section or a page table.  This replaces and entirely deprecates
PAGEOP (finally).  Both Alex Ionescu and Travis Geiselbrecht suggested something similar.

We use fields in the MMPFN to store associations with section offsets, using some dirty
tricks.  We actually store the address of a page table element in a section page table,
along with 8 bits of offset and sync the size of the page table slices to match. The
upshot is that we don't have to have a section page mapped anymore to keep it around, and
we also don't have to track page sharing seperately, instead just using the builtin
reference count, so the previous share and unshare functions in section are now fully
deprecated.  This allows us to cache more (invisible) parts of heavily used files.

Some test code is in here that needs to be cleaned up, such as the indecision exhibited
in MiWaitForBalancer.

Happy new year.

Added:
    branches/arty-newcc/ntoskrnl/mm/reqtools.c   (with props)
    branches/arty-newcc/ntoskrnl/mm/section/fault.c   (with props)
Modified:
    branches/arty-newcc/ntoskrnl/include/internal/newmm.h
    branches/arty-newcc/ntoskrnl/mm/anonmem.c
    branches/arty-newcc/ntoskrnl/mm/balance.c
    branches/arty-newcc/ntoskrnl/mm/elf.inc.h
    branches/arty-newcc/ntoskrnl/mm/elf32.c
    branches/arty-newcc/ntoskrnl/mm/freelist.c
    branches/arty-newcc/ntoskrnl/mm/marea.c
    branches/arty-newcc/ntoskrnl/mm/mmfault.c
    branches/arty-newcc/ntoskrnl/mm/mminit.c
    branches/arty-newcc/ntoskrnl/mm/pagefile.c
    branches/arty-newcc/ntoskrnl/mm/pe.c
    branches/arty-newcc/ntoskrnl/mm/pool.c
    branches/arty-newcc/ntoskrnl/mm/ppool.c
    branches/arty-newcc/ntoskrnl/mm/procsup.c
    branches/arty-newcc/ntoskrnl/mm/region.c
    branches/arty-newcc/ntoskrnl/mm/rmap.c
    branches/arty-newcc/ntoskrnl/mm/rpoolmgr.h
    branches/arty-newcc/ntoskrnl/mm/section.c
    branches/arty-newcc/ntoskrnl/mm/section/data.c
    branches/arty-newcc/ntoskrnl/mm/section/image.c
    branches/arty-newcc/ntoskrnl/mm/section/io.c
    branches/arty-newcc/ntoskrnl/mm/section/pagefile.c
    branches/arty-newcc/ntoskrnl/mm/section/physical.c
    branches/arty-newcc/ntoskrnl/mm/section/sptab.c
    branches/arty-newcc/ntoskrnl/mm/sysldr.c

[This mail would be too long, it was shortened to contain the URLs only.]

Modified: branches/arty-newcc/ntoskrnl/include/internal/newmm.h
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/include/inte...

Modified: branches/arty-newcc/ntoskrnl/mm/anonmem.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/anonmem.c...

Modified: branches/arty-newcc/ntoskrnl/mm/balance.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/balance.c...

Modified: branches/arty-newcc/ntoskrnl/mm/elf.inc.h
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/elf.inc.h...

Modified: branches/arty-newcc/ntoskrnl/mm/elf32.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/elf32.c?r...

Modified: branches/arty-newcc/ntoskrnl/mm/freelist.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/freelist....

Modified: branches/arty-newcc/ntoskrnl/mm/marea.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/marea.c?r...

Modified: branches/arty-newcc/ntoskrnl/mm/mmfault.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/mmfault.c...

Modified: branches/arty-newcc/ntoskrnl/mm/mminit.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/mminit.c?...

Modified: branches/arty-newcc/ntoskrnl/mm/pagefile.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/pagefile....

Modified: branches/arty-newcc/ntoskrnl/mm/pe.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/pe.c?rev=...

Modified: branches/arty-newcc/ntoskrnl/mm/pool.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/pool.c?re...

Modified: branches/arty-newcc/ntoskrnl/mm/ppool.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/ppool.c?r...

Modified: branches/arty-newcc/ntoskrnl/mm/procsup.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/procsup.c...

Modified: branches/arty-newcc/ntoskrnl/mm/region.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/region.c?...

Added: branches/arty-newcc/ntoskrnl/mm/reqtools.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/reqtools....

Modified: branches/arty-newcc/ntoskrnl/mm/rmap.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/rmap.c?re...

Modified: branches/arty-newcc/ntoskrnl/mm/rpoolmgr.h
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/rpoolmgr....

Modified: branches/arty-newcc/ntoskrnl/mm/section.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/section.c...

Modified: branches/arty-newcc/ntoskrnl/mm/section/data.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/section/d...

Added: branches/arty-newcc/ntoskrnl/mm/section/fault.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/section/f...

Modified: branches/arty-newcc/ntoskrnl/mm/section/image.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/section/i...

Modified: branches/arty-newcc/ntoskrnl/mm/section/io.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/section/i...

Modified: branches/arty-newcc/ntoskrnl/mm/section/pagefile.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/section/p...

Modified: branches/arty-newcc/ntoskrnl/mm/section/physical.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/section/p...

Modified: branches/arty-newcc/ntoskrnl/mm/section/sptab.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/section/s...

Modified: branches/arty-newcc/ntoskrnl/mm/sysldr.c
URL: http://svn.reactos.org/svn/reactos/branches/arty-newcc/ntoskrnl/mm/sysldr.c?...