Re: [ros-svn] [ion] 15492: Nonpaged Pool Liberation Day: Allow PagedPool to be used earlier, allow fast mutex to be used earlier on debug builds. Allocate all Se stuff from PagedPool, set the right object types to use paged pool, allocate all strings from paged pool, allocate PE sections from paged pool, and a bunch of other things which should, imo, be in paged pool. If anyone has any contradicting proof, let me know...until then, enjoy ~4-6MB more NonPagedPool
ion@svn.reactos.com wrote:
Nonpaged Pool Liberation Day: Allow PagedPool to be used earlier, allow fast mutex to be used earlier on debug builds. Allocate all Se stuff from PagedPool, set the right object types to use paged pool, allocate all strings from paged pool, allocate PE sections from paged pool, and a bunch of other things which should, imo, be in paged pool. If anyone has any contradicting proof, let me know...until then, enjoy ~4-6MB more NonPagedPool
Hi,
I don't like the exchange from NonPagedPool to PagePool at the current state of ros. There are two reasons:
A) The allocation of paged pool is very slowly compared with the non paged pool allocation.
B) Since the change to the new object type structures, ros has a big memory leakage in the memory (pool) allocation/deallocation of objects. For the non paged pool, there exist some debug functions to find the problem. The paged pool doesn't implement anything to find such problems.
- Hartmut
Hartmut Birr wrote:
ion@svn.reactos.com wrote:
Nonpaged Pool Liberation Day: Allow PagedPool to be used earlier, allow fast mutex to be used earlier on debug builds. Allocate all Se stuff from PagedPool, set the right object types to use paged pool, allocate all strings from paged pool, allocate PE sections from paged pool, and a bunch of other things which should, imo, be in paged pool. If anyone has any contradicting proof, let me know...until then, enjoy ~4-6MB more NonPagedPool
Hi,
I don't like the exchange from NonPagedPool to PagePool at the current state of ros. There are two reasons:
A) The allocation of paged pool is very slowly compared with the non paged pool allocation.
I am aware it's a bit slower (it shouldn't be noticeable), but do we really need to store the Bootup SID, ACL and SDs in *nonpaged* memory? Since when is security a top-priority for speed? Same thing for strings. Windows doesn't even let you RtlCreate strings in non-paged...it's a hack we created because our paged pool memory didn't work early enough.
The only reasons those allocations were ever done in nonpaged pool is because it was impossible to do them in paged pool earlier, but I've fixed that.
B) Since the change to the new object type structures, ros has a big memory leakage in the memory (pool) allocation/deallocation of objects. For the non paged pool, there exist some debug functions to find the problem. The paged pool doesn't implement anything to find such problems.
I can try to quickly hack up a new debug function to dump the Paged Pool (doesn't one exist?). I disagree with hacking the *whole tree* for *everyone* just so that some developers can do debugging. I still have them set to nonpaged locally, just like I have some stuff //#define NDEBUG locally to help me debug problems..but that doesn't mean that everyone should be penalized.
After my changes, I can now boot ROS with only 16MB of Physical Memory. I think it's more important then allocating a security ACL 0.0001 seconds faster... if you think otherwise, I'm open to discussion.
- Hartmut
Best regards, Alex Ionescu
B) Since the change to the new object type structures, ros has a big memory leakage in the memory (pool) allocation/deallocation of objects. For the non paged pool, there exist some debug functions to find the problem. The paged pool doesn't implement anything to find such problems.
- Hartmut
_
By the way:
1) Object Types are still allocaged in NP Pool. So are 99% of objects (except Sections...) 2) I've now implemented tagging for the Object Types so it should be easier to debug.
Best regards, Alex Ionescu
By the way:
- Object Types are still allocaged in NP Pool. So are 99% of objects
(except Sections...) 2) I've now implemented tagging for the Object Types so it should be easier to debug.
I can see the following lines frome the non paged pool statistics at the end of compiling ros on ros:
... Tag 656c6946 (File) Blocks 3172 Total Size 549816 Average Size 173 ... Tag 6e657645 (Even) Blocks 12938 Total Size 793144 Average Size 61 ... Tag 6d4e624f (ObNm) Blocks 28070 Total Size 747432 Average Size 26 Tag 4943624f (ObCI) Blocks 26378 Total Size 1405552 Average Size 53
Now I know that something is wrong with events. But I have no feeling which are the other 13,000 objects.
- Hartmut
Hartmut Birr wrote:
By the way:
- Object Types are still allocaged in NP Pool. So are 99% of objects
(except Sections...) 2) I've now implemented tagging for the Object Types so it should be easier to debug.
I can see the following lines frome the non paged pool statistics at the end of compiling ros on ros:
... Tag 656c6946 (File) Blocks 3172 Total Size 549816 Average Size 173 ... Tag 6e657645 (Even) Blocks 12938 Total Size 793144 Average Size 61 ... Tag 6d4e624f (ObNm) Blocks 28070 Total Size 747432 Average Size 26
Name Information.
Tag 4943624f (ObCI) Blocks 26378 Total Size 1405552 Average Size 53
Create Information.
Now I know that something is wrong with events. But I have no feeling which are the other 13,000 objects.
Looks like the objects aren't being dereferenced properly. Autochk.exe is still alive after boot as well.
Best regards, Alex Ionescu
- Hartmut
Ros-dev mailing list Ros-dev@reactos.com http://reactos.com:8080/mailman/listinfo/ros-dev
Alex Ionescu wrote:
Looks like the objects aren't being dereferenced properly. Autochk.exe is still alive after boot as well.
I've some some testing:
1) Create ExEvent with Nt Function 2) Reference by handle 3) Print New NP Allocs + RefCount 4) Deref 5) Print RefCount 6) Close Handle 7) Print RefCount + Messages in ObpDeleteObject:
(ex/init.c:561) Creating event ******* Dumping non paging pool contents ****** Size 0x28 Tag 0x656e6f4e (None) Allocator 0x80048a45 => The 'hack' for creating the name. My next patch in my WC does this properly, I will probably commit it in the next 2-3 hours or so. Size 0x50 Tag 0x6e657645 (Even) Allocator 0x80049982 => The actual Event (ex/init.c:584) Refcount of New Event: 2 Header: 8039bd40 => After the referencing by handle (ex/init.c:588) Refcount of New Event: 1 => After the ObDeRef (ob/object.c:877) ObpDeleteObject(Header 8039bd58) => During Ntclose (ob/object.c:899) Clearing name => The 0x28 block is freed. (ob/object.c:925) Freeing header at :8039bd40 => The 0x50 block is freed (ex/init.c:592) Refcount of New Event: cccccccc => After the NT Close. The header has been de-allocated
So no leaks on this path =(.
Best regards, Alex Ionescu
Hartmut Birr wrote:
Now I know that something is wrong with events. But I have no feeling which are the other 13,000 objects.
Sections...try editing mm/section.c to be sure.
- Hartmut
Best regards, Alex Ionescu
Hartmut Birr wrote:
By the way:
- Object Types are still allocaged in NP Pool. So are 99% of objects
(except Sections...) 2) I've now implemented tagging for the Object Types so it should be easier to debug.
I can see the following lines frome the non paged pool statistics at the end of compiling ros on ros:
... Tag 656c6946 (File) Blocks 3172 Total Size 549816 Average Size 173 ... Tag 6e657645 (Even) Blocks 12938 Total Size 793144 Average Size 61 ... Tag 6d4e624f (ObNm) Blocks 28070 Total Size 747432 Average Size 26 Tag 4943624f (ObCI) Blocks 26378 Total Size 1405552 Average Size 53
Now I know that something is wrong with events.
One third of the events comes from the 'master' critical section from ntdll. It is only one event per process and not all processes don't remove this event.
- Hartmut
-
Alex Ionescu -
Hartmut Birr