Re: [ros-dev] Guess who's back

[Steven Edwards]

Hi,

--- Phillip Susi <psusi(a)cfl.rr.com> wrote: 
>   Are you talking about being able to create usable ram
> disks at runtime 
>  and use them for general file storage instead of holding the system 
>  volume?  I don't really see any use for ram disks other than holding the 
>  system volume so the media used to boot the system can be removed.  
Yes thats the idea. We tend to get requests for one every so often though I don't know
what people
use them for. There is a example driver with source code on support.microsoft.com that I
always
point people to.

Thanks
Steven



		
__________________________________ 
Yahoo! Messenger 
Show us what our next emoticon should look like. Join the fun. 
http://www.advision.webevents.yahoo.com/emoticontest

[Phillip Susi]

Usually people who want one seem to want it so they can put the pagefile 
on a ramdisk, which is just silly.  Better to explain to them why that's 
a silly idea then they won't want a ramdisk anymore ;)


Steven Edwards wrote:

>  Hi,
>
> --- Phillip Susi <psusi(a)cfl.rr.com> wrote: 
>   
>
> >  Are you talking about being able to create usable
> > ram disks at runtime 
> > and use them for general file storage instead of holding the system 
> > volume?  I don't really see any use for ram disks other than holding the 
> > system volume so the media used to boot the system can be removed. 
> >     
> >  
> Yes thats the idea. We tend to get requests for one every so often though I don't know
> what people
> use them for. There is a example driver with source code on support.microsoft.com that I
> always
> point people to.
>
> Thanks
> Steven
>
>
>
> 		
> __________________________________ 
> Yahoo! Messenger 
> Show us what our next emoticon should look like. Join the fun. 
> http://www.advision.webevents.yahoo.com/emoticontest
> _______________________________________________
> Ros-dev mailing list
> Ros-dev(a)reactos.com
> http://reactos.com:8080/mailman/listinfo/ros-dev
>
>   
>

[Richard Campbell]

ACTUALLY if you have the page file disabled it causes compatibility 
problems with apps.  Windows is DESIGNED to have a page file.  So rather 
than keeping a page file on disk...create it on a RAMDrive and you keep 
apps happy and windows happy.

Richard

Phillip Susi wrote:

>   Usually people who want one seem to want it so they
> can put the 
>  pagefile on a ramdisk, which is just silly.  Better to explain to them 
>  why that's a silly idea then they won't want a ramdisk anymore ;)
>
>
>  Steven Edwards wrote:
>
> >   Hi,
> >
> >  --- Phillip Susi <psusi(a)cfl.rr.com> wrote:  
> >
> > >   Are you talking about being able to create usable
> > > ram disks at 
> > >  runtime and use them for general file storage instead of holding the 
> > >  system volume?  I don't really see any use for ram disks other than 
> > >  holding the system volume so the media used to boot the system can 
> > >  be removed.    
> >
> >  Yes thats the idea. We tend to get requests for one every so often 
> >  though I don't know what people
> >  use them for. There is a example driver with source code on 
> >  support.microsoft.com that I always
> >  point people to.
> >
> >  Thanks
> >  Steven
> >
> >
> >
> >         
> >  __________________________________ Yahoo! Messenger Show us what our 
> >  next emoticon should look like. Join the fun. 
> >  http://www.advision.webevents.yahoo.com/emoticontest
> >  _______________________________________________
> >  Ros-dev mailing list
> >  Ros-dev(a)reactos.com
> >  http://reactos.com:8080/mailman/listinfo/ros-dev
> >
> >   
> >  
>  _______________________________________________
>  Ros-dev mailing list
>  Ros-dev(a)reactos.com
>  http://reactos.com:8080/mailman/listinfo/ros-dev
>
>

[Phillip Susi]

Could you be any more specific?  Applications should not know or care 
about pagefiles, that's the business of the memory manager.  Most people 
have a misconception that if you have a gig of ram, it is a good thing 
to allocate 512 megs to a ramdisk and put a pagefile there, when in 
fact, it is better to simply have no pagefile at all and have a full gig 
of usable ram.  I say this because the entire reason for having a 
pagefile is to store data that can't fit into real ram, so by reserving 
half your ram for a ramdisk, you are creating the need for the pagefile 
in it.  If you leave that 512 megs of ram to the vm system to manage, it 
won't even need a pagefile since there will be a gig of physical ram to 
work with instead of only 512 megs.

Also you have to remember that most of the time, the pagefile is not 
used, it is only there as a last resort reserve pool of memory.  Unless 
the system is very heavily loaded, it isn't going to be used at all, so 
when you reserve 512 megs of ram for a ramdisk to hold the pagefile, 
most of that memory goes to waste.  Really even under maxium total 
commit, you still are not going to use the pagefile fully used because a 
good deal of virtual memory in the system is used to map shared physical 
pages of ram for things like dlls.  Each time the dll is mapped, it uses 
up virtual memory but only physical memory once, so when you map a 1 meg 
dll 20 times, you've used 20 megs of virtual memory, but only 1 meg of 
real memory.  Thus if you have 10 megs of ram and 10 megs of pagefile, 
even though you have 20 megs of used virtual memory, you are only using 
1 meg of ram, zero space in the pagefile, and have 9 megs of free ram ( 
though zero free virtual memory so more allocations will fail ). 

It's kind of like having an application that has its own file caching.  
It's using up memory to cache files that are already cached by the 
kernel cache manager, and using up memory that the kernel is better at 
allocating than the application is, so it's best to just let the kernel 
worry about caching and not the application. 



Richard Campbell wrote:

>   ACTUALLY if you have the page file disabled it causes
> compatibility 
>  problems with apps.  Windows is DESIGNED to have a page file.  So 
>  rather than keeping a page file on disk...create it on a RAMDrive and 
>  you keep apps happy and windows happy.
>
>  Richard
>
>  Phillip Susi wrote:
>

[Richard Campbell]

I've used several applications that CHECK to see if the pagefile is 
active, if not, they refuse to load.  This mostly happens with games.  
Early in the beta world of warcraft was like that (even though they 
removed the check in later betas/final.)  Also, even when you have 'more 
than enough memory',  seldom 'used' data is moved to a page file when it 
isn't being accessed.  If i have a chance i'll find some links for you. 
(mostly Windows NT stuff though, don't have anything on other operating 
systems, but then again that's what matters here.)

Windows also tends to act rather funny when you don't have a page file 
(keep in mind, i'm using a machine with 3 gb memory.  My page file is on 
my hd, having one in RAM to me is nonsense because my pagefile is 
generally 3-6 gigs and would outgrow RAM very fast if i did that.
Of course i'm not recommending creating a ramdrive just for a page file, 
but i'm making it clear that it isn't totally foolhardy.

Phillip Susi wrote:

>   Could you be any more specific?  Applications should
> not know or care 
>  about pagefiles, that's the business of the memory manager.  Most 
>  people have a misconception that if you have a gig of ram, it is a 
>  good thing to allocate 512 megs to a ramdisk and put a pagefile there, 
>  when in fact, it is better to simply have no pagefile at all and have 
>  a full gig of usable ram.  I say this because the entire reason for 
>  having a pagefile is to store data that can't fit into real ram, so by 
>  reserving half your ram for a ramdisk, you are creating the need for 
>  the pagefile in it.  If you leave that 512 megs of ram to the vm 
>  system to manage, it won't even need a pagefile since there will be a 
>  gig of physical ram to work with instead of only 512 megs.
>
>  Also you have to remember that most of the time, the pagefile is not 
>  used, it is only there as a last resort reserve pool of memory.  
>  Unless the system is very heavily loaded, it isn't going to be used at 
>  all, so when you reserve 512 megs of ram for a ramdisk to hold the 
>  pagefile, most of that memory goes to waste.  Really even under maxium 
>  total commit, you still are not going to use the pagefile fully used 
>  because a good deal of virtual memory in the system is used to map 
>  shared physical pages of ram for things like dlls.  Each time the dll 
>  is mapped, it uses up virtual memory but only physical memory once, so 
>  when you map a 1 meg dll 20 times, you've used 20 megs of virtual 
>  memory, but only 1 meg of real memory.  Thus if you have 10 megs of 
>  ram and 10 megs of pagefile, even though you have 20 megs of used 
>  virtual memory, you are only using 1 meg of ram, zero space in the 
>  pagefile, and have 9 megs of free ram ( though zero free virtual 
>  memory so more allocations will fail ).
>  It's kind of like having an application that has its own file 
>  caching.  It's using up memory to cache files that are already cached 
>  by the kernel cache manager, and using up memory that the kernel is 
>  better at allocating than the application is, so it's best to just let 
>  the kernel worry about caching and not the application.
>
>
>  Richard Campbell wrote:
>
> >   ACTUALLY if you have the page file disabled it
> > causes compatibility 
> >  problems with apps.  Windows is DESIGNED to have a page file.  So 
> >  rather than keeping a page file on disk...create it on a RAMDrive and 
> >  you keep apps happy and windows happy.
> >
> >  Richard
> >
> >  Phillip Susi wrote:
> >  
>  _______________________________________________
>  Ros-dev mailing list
>  Ros-dev(a)reactos.com
>  http://reactos.com:8080/mailman/listinfo/ros-dev
>
>

[Phillip Susi]

Hrm... well, aside from shooting the developers of software that did 
something like that, I think it would be better to have a compatibility 
flag that would have the OS lie to the application about having a 
pagefile rather than dedicating part of ram for a ramdisk to hold a 
pagefile. 

Like you said, the kernel constantly is trying to free up ram by paging 
out memory that is not being used.  If you have a pagefile in ram, then 
it isn't really freeing memory by moving things to the page file, 
instead it is just wasting time moving it from one place in ram to 
another and back again, with no benefit.  Also because most of the page 
file goes unused, a good deal of the ram you have reserved for it will 
be wasted when it could be better put to use by the filesystem cache.  
It would be better for performance to simply not have a pagefile and 
thus not shuffle around ram needlessly.  Since the goal of paging is to 
free up ram, paging to a ramdisk is self defeating. 


Richard Campbell wrote:

>   I've used several applications that CHECK to see
> if the pagefile is 
>  active, if not, they refuse to load.  This mostly happens with games.  
>  Early in the beta world of warcraft was like that (even though they 
>  removed the check in later betas/final.)  Also, even when you have 
>  'more than enough memory',  seldom 'used' data is moved to a page file 
>  when it isn't being accessed.  If i have a chance i'll find some links 
>  for you. (mostly Windows NT stuff though, don't have anything on other 
>  operating systems, but then again that's what matters here.)
>
>  Windows also tends to act rather funny when you don't have a page file 
>  (keep in mind, i'm using a machine with 3 gb memory.  My page file is 
>  on my hd, having one in RAM to me is nonsense because my pagefile is 
>  generally 3-6 gigs and would outgrow RAM very fast if i did that.
>  Of course i'm not recommending creating a ramdrive just for a page 
>  file, but i'm making it clear that it isn't totally foolhardy.
>
>  Phillip Susi wrote:
>  
Royce Mitchell III wrote:

>   Casper and I want one to improve build times, by
> putting certain 
>  temporary files there.  

If the cache manager is doing it's job, temporary files already reside 
in ram without having to dedicate a portion of it to a ramdisk.  If the 
cache manager isn't working at least as well with temporary files as a 
ramdisk is, maybe it could be improved?

[Royce Mitchell III]

Phillip Susi wrote:

>   If the cache manager is doing it's job, temporary
> files already reside 
>  in ram without having to dedicate a portion of it to a ramdisk.  If 
>  the cache manager isn't working at least as well with temporary files 
>  as a ramdisk is, maybe it could be improved?
>  
This is for use in building from Windows, too. These files are not 
flagged as temporary, because in the normal sense they are true files. 
For example, the .o files. Normally a developer keeps these files so 
only what's necessary needs to be rebuilt. However, if you have a 
"nightly build" setup and want to do a complete rebuild every time, then 
it makes sense.

[Phillip Susi]

Hrm... interesting case.  If you are going to check out the repository, 
build it, then delete the files, they never need to be written to disk.  
This case might benefit from a temporary filesystem instead of a 
ramdisk.  I think it was solaris that has a tempfs that is specially 
tailored for temporary files.  It houses them in pagable memory.  Maybe 
we could develop a filesystem like that where it stores files created in 
it in named memory sections.  That way the filesystem can grow and 
shrink as needed, rather than reserving a fixed chunk of ram all the 
time.  The memory sections would also be pagable, so if the system 
really needs the ram, it can still page out the temporary data, but 
otherwise it will all be kept in ram.  It also would handle creating and 
deleting files with less overhead than fat or ntfs. 

Time for me to dig out the NT filesystems book. 

Royce Mitchell III wrote:

>   This is for use in building from Windows, too. These
> files are not 
>  flagged as temporary, because in the normal sense they are true files. 
>  For example, the .o files. Normally a developer keeps these files so 
>  only what's necessary needs to be rebuilt. However, if you have a 
>  "nightly build" setup and want to do a complete rebuild every time, 
>  then it makes sense.
>
>  _______________________________________________
>  Ros-dev mailing list
>  Ros-dev(a)reactos.com
>  http://reactos.com:8080/mailman/listinfo/ros-dev
>

[Jakob Eriksson]

Phillip Susi wrote:

>   Hrm... interesting case.  If you are going to check
> out the 
>  repository, build it, then delete the files, they never need to be 
>  written to disk.  This case might benefit from a temporary filesystem 
>  instead of a ramdisk.  I think it was solaris that has a tempfs that 
>  is specially tailored for temporary files.  It houses them in pagable 
>  memory.  Maybe we could develop a filesystem like that where it stores 
>  files created in it in named memory sections.  That way the filesystem 
>  can grow and shrink as needed, rather than reserving a fixed chunk of 
>  ram all the time. 

Like the AmigaOS RAM disk ... ;-)


//Jakob