Re: registry TODO

[Boaz Harrosh]

James Hawkins wrote:

>  >Nobody's working on it, so it won't be
> supported until someone cares
> >enough to do it. I encouraged a few people to start working on it but
> >nobody did, so taking out the existing support is a way to provide
> >more encouragement. If that's not enough then the feature simply won't
> >be supported anymore.
> >    
> > 
I started toying around the current and old code, Gathering knowledge 
and ideas. What I think of doing is:
  keep the boot registry as is (more less). Put in on wine/wine/config a 
key system that enables a "plugin" system of registry loaders/savers. 
Builtin plugins are current format and Windows binary format. Other 
formats could be easily implemented; candidates could be XML, MySQL etc...
The system I thought of would also save the hives and sub trees, to the 
files/plugins it was loaded from. Each plugin could also implement a 
write behind so Data don't get lost in case of a crash or power lost.

Good point with the on demand-loading/cache of keys. It should be done 
this way. So in summary:
1. Implement a Memory registry with plugin drivers and mount semantics. 
This driver goes here, that driver mounts there.
2. Implement 2 basic drivers: Wine builtin format, Windows binary format.
3. Enable dynamic load of external drivers.

The Initial registry system is always loaded from system.reg builtin 
format which is also the "fstab" file of the registry. At minimum every 
thing is exactly like now one (two) files builtin format, and that's it. 
At the maximum it can be highly object oriented with per user, per 
application, per group management.

Please tell me what you think.
Any libraries I should look into?
How is ReactOS's current (future) system?

Free Life
Boaz

[Thomas Weidenmueller]

Boaz Harrosh wrote:

>   Good point with the on demand-loading/cache of keys.
> It should be done 
>  this way. So in summary:
>  1. Implement a Memory registry with plugin drivers and mount 
>  semantics. This driver goes here, that driver mounts there.
>  2. Implement 2 basic drivers: Wine builtin format, Windows binary format.
>  3. Enable dynamic load of external drivers. 
ReactOS has (almost) all that's needed. However, a driver for registries 
is impossible because the registry has to be present before any driver 
was loaded. Our freeldr as well has to load the registry in order to be 
able to determine which boot drivers to load. Supporting wine's builtin 
format (i guess you mean the text based registry) is not appreciated and 
I think the architecture, as it is right now, is most appreciated.

>   How is ReactOS's current (future) system? 
See above

Best Regards,
Thomas

[Mike Nordell]

If it seems I come down a bit hard on the proposal, I can only blame
language barriers. My intention is only to illuminate some of the pitfalls I
think are inherent in a dynamic load system as the proposed.

Boaz Harrosh wrote:

>   What I think of doing is:
>    keep the boot registry as is (more less). 
This is, I believe, not an area open for discussion. It is as it is. If this
is correct, then basically all of HKLM\SYSTEM (and linked keys) are
off-limit.

>   Put in on wine/wine/config a
>  key system that enables a "plugin" system of registry loaders/savers. 
A couple of questions:

SAM? How is security supposed to be handled using this approach - assuming
SAM is to handle the security (which I must assume, else I'm against this)?

Links? Especially links pointing to another "handler".

Volatile keys?

>   Builtin plugins are current format and Windows binary
> format. Other
>  formats could be easily implemented; candidates could be XML, MySQL etc... 
Having seen the amount of registry lookups the NT5.0 (aka Windows 2000)
Explorer does just for viewing a single directory, I suspect running that
over an SQL client to a server (or even parsing a local XML file) one might
be in for a treat making making Windows XP on Bochs, on a 486, look fast.

I might also point out the fact that registry-handling is currently done
entirely from kernel mode, with unrestricted access rights. Would you trust
an XML parser or an SQL client these unrestricted rights? I know for certain
I wouldn't.

One could, theoretically, solve this by having such things run as services
(in user-mode, and under restricted accounts), and have the kernel call out
to them using RPC (how to establish trust between the kernel and such
serices? LPC is out of the question I think) - opening the door for
effectively making a system look hung if one of the servers this would
depend upon was unavailable. Again, think Explorer.exe and its gazillion
registry accesses.

>   The system I thought of would also save the hives and
> sub trees, to the
>  files/plugins it was loaded from. 
If a plugin has started handling a key, it would (obviously) have to handle
all sub-keys too - even if delegating deeper down.

>   Each plugin could also implement a
>  write behind so Data don't get lost in case of a crash or power lost. 
If this is to be a serious proposal, it would have to provide loggin
mechanisms. As it would be basically impossible to hold data consistency
across networks, SQL servers and stuff, the data would have to be stored in
a log on the boot volume (if for nothing else, to assure correct order of
updates).

>   Good point with the on demand-loading/cache of keys.
> It should be done
>  this way. 
It might seem reasonable at first, but saying it _should_ be this way
without any data to back it up seems a bit premature. I haven't seen any
empirical data to back it up, and no testing for what happens when e.g. any
other part of the system encounters a catastrophic failure (the "system" is
now extended to include network, routers and even remote SQL servers).

I suspect (with a bit of experience in the area) that this might actually be
completely unsatisfactory, and that it in fact should grab the whole hive(s)
from the remote storage, storing it/them locally, to allow for network
outages without hanging the whole system or crashing apps.

>   So in summary:
>  1. Implement a Memory registry with plugin drivers and mount semantics. 
Please define "Memory registry". Please keep in mind that the registry on
Windows machines are nowadays (especially if you install .NET) of such a
size as to require more storage than you needed RAM to boot NT4, why I
suspect it needs to be swappable/discardable.

>   This driver goes here, that driver mounts there.
>  2. Implement 2 basic drivers: Wine builtin format, Windows binary format. 
I don't know what "Wine builtin format" is, neither do I know how it fits
the NT security model. Could you elaborate?

>   3. Enable dynamic load of external drivers. 
If this is to be handled as suggested, by a registry branch below
wine/wine/config, it means that branch would have to be locked down to only
allow administrator write/modify (maybe even read) access. It also opens the
door for serious malice, in case a malicious service registers the RPC
service before one of the intended services got to do its work, and...

Basically, I urge you to seriously reconsider your proposal while thinking
of every aspect you can imagine (and then a few more). I think the
intentions are good, and I like the basic ideas, but at the same time I
suspect the effort and all the implications might have been underestimated.
Especially the security problems.

>   The Initial registry system is always loaded from
> system.reg builtin
>  format which is also the "fstab" file of the registry. At minimum every
>  thing is exactly like now one (two) files builtin format, and that's it.
>  At the maximum it can be highly object oriented with per user, per
>  application, per group management. 
I assume you know how MS handles this, by downloading a roaming registry
from a DC to the local machine? Knowing the unreliability of networks, and
other non-local media (compared to local media, even if that fails too) I
nowadays have grown to think MS did the right choice in having the hives
downloaded to the local machine when logging on.


/Mike

[Boaz Harrosh]

Mike Nordell wrote:

>  If it seems I come down a bit hard on the proposal, I
> can only blame
> language barriers. My intention is only to illuminate some of the pitfalls I
> think are inherent in a dynamic load system as the proposed.
>
>   
>  
I apologize again. The intended proposal was meant for Wine only. I 
cc'ed ReactOS because there was a Question about ReactOS at the end. I 
wanted to know if there was some codebase I could use. So the proposal 
is not at all relevant to ReactOS Kernel registry! ;)

You did make good points about speed and Security. A few are relevant to 
Wine, and I will take them into consideration. (For Wine).

By the way from tests I made on Windows some of the XML implementations 
(DOM style, load everything to memory) are faster than Registry 
functions. I suspect that is do to hash table size, and User-Kernel land 
switching. but it is in the same time frame.
And... "dynamic load system"s is not a new subject. It is just the same 
as a file system, with a mounting table and all that. Even the very root 
uses a driver stack (Some what hardcoded) and lots of Unix systems know 
how to switch roots mid-flight....If you ask me I'd put all that 
Registry hell directly into the file system. Why do we need a duplicate 
implementation. It is already dynamic, driver driven, secure, cached, 
Journaled, optimized to death...But Microsoft made that mistake in the 
past and we most follow.


Sorry about the miss-understanding
Free Life
Boaz

[Jakob Eriksson]

Boaz Harrosh wrote:

>   to switch roots mid-flight....If you ask me I'd
> put all that Registry 
>  hell directly into the file system. Why do we need a duplicate 
>  implementation. It is already dynamic, driver driven, secure, cached, 
>  Journaled, optimized to death...But Microsoft made that mistake in the 
>  past and we most follow. 

Isn't ReiserFS (especially 4 and later) developed exactly for these things?
To use the filesystem directly as the database.
AFAIK Reiser has some grand scheme and the filesystem is the foundation 
for that.

regards,
Jakob