2 Sep
2005
2 Sep
'05
12:39 a.m.
[please note: due to its cross-discipline and cross-project nature,
this message is going out to SEVERAL related project mailing lists.
when replying, please take EXTRA caution not least because some of
these lists are subscriber-only. also, please note: i _would_
post this also to the samba mailing lists but due to the fascist
censorship in place since the 15th dec 2004, i am unable to do so.
this is their decision and it is their loss. i am not asking you
to respect that decision i am simply making you aware of it.]
hi,
out the woodwork i pop - not necessarily ready to chew anything because
i know just how much work's involved, but what i did want to do was
say "hi, i'm still here" and do a brain-dump of how authentication
ideally needs to be implemented in reactos.
at 2,600 words and 16k, this message is quite long and so i have
placed a copy at http://lkcl.net/software/reports/reactos-auth.txt,
just in case it doesn't make it past various mailing-list limits
(i'll find out in a couple of minutes... :)
it breaks down into a number of sections:
1) i describe the timescales and ways to cut them,
along with some warnings and stuff.
2) amonst other stuff i outline some background as to
why i am posting this to so many lists.
3) i outline a project plan and the dependencies
and "optional" steps,
4) i describe a recommended implementation
roadmap starting with the "minimum" requirements,
and expand on some technical info and references
i found, which would help with some of the
"nice-to-have" steps.
so. first.
please do not be scared by how much work is involved, and how much code
there is. it all hinges one ONE function and that one function... you
are _not_ going to believe how much code that one function drags in,
kicking and screaming, behind it.
please also please i beg you DO NOT consider going "bleedin 'ellfire,
that's so much frakkin work we can't POSSIBLY do it the way you
describe, we MUST do it our own way, starting with what we know, love
and have already started tackling, and can't possibly back out of what
we've already done".
should you choose to exercise the "NIH" option, i frakkin
guarantee you that you will waste about five to eight years
of your (collective) lives reinventing the wheel.
"The Plan" outlined here will shave that down to about 12 to 18 months,
utilising some SIX HUNDRED THOUSAND lines of pre-existing code, and
later in this message i also outline a prioritisation of the necessary
work to "cut down" the time to maybe about ... mmm... three or so
months, by leaving out some of the "nice-to-have" stuff. actually...
_all_ of the nice-to-have stuff :)
that will at least "GetItWorking(tm)" and the rest of the bits
can be considered at leisure once people go "god that's awful,
we can't possibly leave it like that" and hopefully hopefully
things will actually progress from there.
remember - please: this stuff is sufficiently complicated such that
you really can't afford the niceties such as "It Must Be Perfect (tm)"
before it can be accepted. i've seen that shit before, and it's
a baad mxxxxxxxer path to go down, especially with such complex
and heavily interdependant reverse-engineering projects as reactos,
wine, samba and freedce.
anyway.
fyi - before i begin, i should mention a couple of things:
1) this message also goes out the the apache devel mailing
list (specifically the APR one) because the NT style
authentication thing is the sort of thing that really _should_
be in a (special?) APR util library, along with "NT Named Pipes" -
see http://en.wikipedia.org/wiki/NamedPipes
one of the reasons why NT-style NamedPipes is _not_ in an APR util is
because it is believed that NT-style NamedPipes do not fit the
"least common denominator". by having the infrastructure outlined
below, it is possible to move things "up one level" to the point
where unix _has_ that denominononmination.
also, if APR still has support for that god-awful program-running
program called Win95, it would, with not much extra effort, be
possible to port some reactos components to Win95 (!) such that
ThePlan outlined here would make Win95 have proper NT-style
authentication (!) now there's a horrible thought that will give
MS and free software people nightmares both: upgrading Win95 by
installing free software components at it. muhahahahah ahem.
2) i have filled in a number of pages on wikipedia.org. see
http://en.wikipedia.org/wiki/User:lkcl. please take the time to
review these pages, PLEASE help me with my totally biased "POV"
(point-of-view) comments, by either editing the pages direct or by adding
comments on the "talk" pages as appropriate. or alternatively dumping
me in the nearest pond if ever you meet me.
wikipedia is supposed to be encyclopedia-ic and some of my
comments are anything but that.
help!!!
3) please due to the quite broad distribution of this message across
multiple mailing lists, please do NOT ask stupid questions like
"what the hell do we need to use samba tng for, why can't we use samba
in reactos"? and "why is this guy bothering us with this insane stuff?"
and "how do i use killfiles?" or terry pratchett's - more specifically
the librarian's favourite - "where has all the pie and custard gone?"
please do your research: see if nothing else
http://en.wikipedia.org/wiki/Samba_TNG_software and then start googling.
anyway.
onward.
roughly in order of dependence, with "nice-to-have" status added as
well:
1) port FreeDCE to Win32 - more specifically add in autoconf support for an
alternative threading model - the one in reactos. no, this is not a joke:
MSRPC is a critical part of NT infrastructure it's just that very few people
are actually aware of this (including the US DoJ and the EU commission...)
see http://www.google.co.uk/search?q=lkcl+wine+freedce for discussion
links as to why - it's a long story and it took me several dozen
messages spanning over many days to outline it in sufficient detail
for it to be understood (mostly me getting things straight in my
head...)
status: "nice to have" but you will soon wish that you _did_ have it!!!
there is also an opportunity to support Wine, here: essentially
it's the same job, for DCOM, for proper "authentication"
purposes in Wine just like in ReactOS... it's a long story.
2) remove dependence on "unix" security model from Samba TNG's services
starting with samrd (perhaps by finishing off samrtdbd?)
status: "essential".
sub-projects:
a) in Samba TNG, rework winbindd such that it is a REQUIRED service,
and it has "modes" that, instead of "inventing" links between
unix uids and nt sids, it reads smbpasswd and correlates unix uids
with nt sids _just_ like is done now - but hard-coded through
the awful code that has had to dieee since... about two weeks after
i wrote it, back in 1999.
this one's a long story but it's not actually required for
reactos but _might_ be required for wine...
status: "pretty much essential" but not for reactos but for making life
easy to share tng development between reactos and unix.
b) in Samba TNG, make the authentication code (smb-side and rpc server
back-end-wise) contact winbindd BUT ONLY for resolving unix uids and
unix gids - which is actually a VERY small task involving a few
hundred lines of code.
this goes back to the "SURS" stuff i wrote up in 1999 - but instead
of a library it hands off the responsibility for sid<->uid+gid
lookups to winbindd.
it _may_ also be essential for Wine to operate correctly
with authentication (which is a "stub" at the moment just
like it is in reactos), to interoperate with other programs,
to cache login credentials in order to do smb client-side
operations, that sort of thing, under which circumstances,
it will be necessary for Wine to have the same sort of thing.
it's no big deal :)
status: same as for 2a).
c) in Samba TNG, abandon all use of hard-coded MSRPC stuff and utilise
FreeDCE instead, and do a rewrite of all services, one by one
(and because samba tng is modular, this _can_ be done one-by-one,
checking MSRPC interoperability along the way between the "old"
services, the "new" services, and also with NT itself).
in and of itself, this critically depends on making
DCEThreads reliable, though, or adding support (finally!) for
POSIX threads.
status: "non-essential" but you would soon end up wishing that you had :)
3) complete LsaLogonUser, add LsaLogonUserEx, in NTOSKRNL.DLL, and
friends.
my favourites. the LSASS stuff.
these functions are quite simple: they are "redirectors" - a vector
table of functions is required to be passed in, which includes things
like "authenticate with my lovely service" and "free some memory".
there are many LSASS sub-services: one of them is Netware, one of
them is MSV1_0.DLL, one of them is Kerberos.
there are others.
LSASS is based on _exactly_ the same technique that dlopen does e.g
in libdvdcss, and in freedce's ntlmsspauth.so, and in... mmm...
the freedce transport module infrastructure and _many_ other
projects ... except of course it's NT-based table of
higher-order-functions not unix "dlopen()" ones.
i'll stop trying to teach people to suck eggs, now *embarrassed* :)
status: "essential".
sub-projects:
a) write an MSV1_0.DLL which registers with the LSASS service.
this will utilise MSRPC functions that call into the Samba TNG
"NETLOGON" service.
there already exists "basic" functions inside Samba TNG that...
well... it's a bit messy, but they work.
the key function to be calling is cli_nt_login_interactive,
and you will notice _very_ quickly from the arguments hey, some
of those look... kinda... familiar!!
see http://viewcvs.samba-tng.org/cgi-bin/viewcvs.cgi/tng/source/rpc_client/cli_…
status: "essential."
the "porting" bit of this code to FreeDCE i would classify as
"non-essential" but again, you will soon wish that you _were_ using
FreeDCE.
that's basically it: there are other sub-projects such as
turning the "Registry" code in ReactOS (or Wine) into an MSRPC
service (yes, i did say pretty much everything that's anything
critical in NT is an MSRPC service, didn't i? :) but they aren't
"essential".
i say basically it, but that ONE stupid function, cli_nt_login_interactive,
drags in quite literally HALF A MILLION lines of code - 250,000 lines of
it in FreeDCE, alone (which, like i said, could possibly be avoided but
the hard-coded MSRPC stuff in samba tng is sufficiently awkward to make
a rewrite utilising FreeDCE very attractive).
what i would suggest is the following:
1) write an MSV1_0.DLL "test stub" in combination with completing the
LSASS functions, which supports a username "test", domain of "test"
and
password of "test".
code up a hard-coded "blob" - a NET_USER_INFO_3 structure - that contains
the response / essential information of groups, gids, sids, user session
key etc.
see http://viewcvs.samba-tng.org/cgi-bin/viewcvs.cgi/tng/source/include/rpc_net…
(note: i _did_ say it would be nice to utilise FreeDCE didn't i? well,
the NET_USER_INFO_3 structure in that header file is a "messy" version that
i recreated from off-the-wire back in about ... mmmm... 1997. there
_do_ exist IDL-generated versions of this data structure, thanks to
matthew chapman - NETLOGON.idl for example).
once that works...
2) write a lovely insecure method of "outsourcing" the username,
domain and password to an external server - Samba TNG - which performs
the authentication on your behalf and gets back "real" data.
this could be done simply with a TCP connection, throw the data
in-the-clear over to a simple temporary shim service blah blah,
bob's your uncle.
3) port samba tng's netlogond, samrd and lsarpcd to ReactOS.
this is quite straightforward: about the only really essential
"missing" bit is to tie the services in to "NT Named Pipes" rather
than unix domain sockets.
_but_ - they only need to be plugged in to a back-end transport
which all services - any of samba tng's MSRPC services - would
use. so there's a key bit of work needed - probably under
400 lines of code - and the rest should fall into place.
this is where it would be SO much easier to be utilising FreeDCE.
all that's needed would be to write a FreeDCE "transport"
plugin for ReactOS - ncacn_np - and then you'd be DONE.
it's a long story...
hey, has someone implemented TransactNamedPipe() and CreateNamedPipe()
in ReactOS?
if not, it's quite straightforward to do, and it involves dropping
opaque data blobs onto smbd (again, smbd ported to reactos...)
i _did_ say it's a long story, didn't i? :)
4) finally: track down how "LsaLogonUserEx" works. LsaLogonUser
utilises cleartext passwords. LsaLogonUserEx i BELIEVE utilises NT and
LM password hashes, or some brain-dead encrypted variant thereof.
once 1) is completed, then TA-DAAAAA!!! lib/secur/lsa.c's "LsaLogonUser"
function actually gets REAL DATA!
okay.
notes:
1) regarding FreeDCE. see http://en.wikipedia.org/wiki/FreeDCE
freedce is an interoperable version of MSRPC that is derived from
EXACTLY the same source code (DCE 1.0 reference implementation) that
is present in Windows NT.
DCE/RPC was co-opted / adopted / borg-ified by microsoft to form
the basis of NT domains because paul leach, a co-founder of
apollo computers and originator of NCS which became DCE/RPC,
ended up working for microsoft back at the time when dave
cutler was doing the original NT 3.1.
using FreeDCE is non-essential for small projects of less than about
8,000 lines of code. Samba TNG's MSRPC code comprises OVER ONE HUNDRED
THOUSAND lines of code.
that i carried on hand-crafting MSRPC packets for three years simply
demonstrates quite how bloody stupid i am.
that you - the wine team - continue to reinvent an non-interoperable
version of MSRPC, for binary-level "DCOM" interoperabiltiy ONLY,
demonstrates quite how just as bloody stupid you are being. that _can_
be taken as a compliment, as i genuinely i mean it with the greatest of
respect.
2) regarding a strategy to "minimise" the amount of time needed to
"Get Things Working".
a) utilise samba tng as-it-is, porting it as-is to ReactOS (mingw32)
b) utilise cli_nt_login_interactive() as-it-is.
c) complete MSV1_0.DLL.
this will be SUFFICIENT and would only take a few months. enhancements
come later, roughly in this order:
a) complete a port of FreeDCE to ReactOS, by beating DCEthreads to
death and replacing it with Win32 (NT) threads - #ifdef style,
autoconf style.
b) locate NETLOGON.idl, samr.idl and lsarpc.idl from matthew chapman's
stuff, or from "todd sabin's" http://razor.bindview.com, or from
the samba web site.
actually... see: http://www.bindview.com/services/razor/utilities/
compile up CLIENT-SIDE ONLY, using dceidl, and create a
library for use inside MSV1_0.DLL.
i know todd developed interoperable versions of these idl files,
because he was using them, compiled with MIDL.EXE, to do security
tests against NT 4.0. remember: these IDL files are what microsoft
DOESN'T want anyone to have, and there are very good _legitimate_
reasons for it: they are "behind-the-scenes" APIs which, if some
idiot inside microsoft went and wrote a tool which became publicly
used and relied on, they would be stuffed: they have ENOUGH apis
which date back 15 years and they don't need any more. that
having been said, i hate the fact that they are forcing people
to pay $50k up-front and then $100 per-client for frakkin
_information_. bastards.
c) replace cli_nt_login_interactive() with a function that utilises
the above library that utilises netlogon, samr and lsarpc client-side
stuff from b) above.
basically, what it boils down to is that the functions inside
cli_nt_login_interactive i wrote BY HAND after examining MSRPC
traffic. those functions neeeed too dieeee and they can easily
be replaced one-by-one with the "proper" versions from doing
"dceidl -client lsarpc.idl". it takes a couple of seconds and
you get a header file lsarpc.h and a lsarpc.o and you're DONE.
question. why did i spend so long doing hand-marshalling of the
nt domains code. perhaps because i would not be able to give
microsoft absolute hell for three frakkin years?
d) consider porting, one-by-one, the Samba TNG services lsarpcd,
netlogond and samrd (or better samrtdbd) to FreeDCE runtime
infrastructure.
this is NOT essential but it is very worthwhile. every project
needs to do at least two throw-aways and with the samba tng
code, there has been approximately one and a half throwaways
so far... time for a major one, ESPECIALLY in light of the reactos
project.
FINALLY:
e) consider writing that winregd service but actually picking up REAL
registry hive files. someone did write a "reader" of registry files,
i do not recall who it was.
[note: YESSSS!!! yeeeeeehawww, todd i could KISS you!!]
http://www.bindview.com/Services/RAZOR/Utilities/Unix_Linux/ntreg_readme.cfm
todd wrote a registry-hive-reader as a linux filesystem driver!!!!!
todd, you are bloody mad. *smooch*.
f) consider writing a samr service with the FreeDCE runtime
that utilises "registry" functions RegXXXX - _properly_.
anyway, if you got this far: congratulations and welcome to a brief
nightmare history of the development behind NT Domains Authentication.
all of the infrastructure above _already_ exists - in Samba TNG.
it's just... it works. it's a bit creaky at the edges, but it works.
how about it?
l.
p.s. it almost goes without saying, but i will hint at it once again:
you will not find the samba team's goals and strategic direction to be
compatible with reactos. as a team, they lack the insight, vision and
project management skills to be able to cope with such inter-dependant
insane gibberish. this i say with much sadness because they are highly
respected individuals in the free software community, and yet they are
taking the samba project in very different - and ultimately
time-consuming - directions from what is really really needed. due
to pride they cannot admit this in public but they have been known
to admit it in private. much much disappointment and sadness and there's
absolutely nothing i can do about it.
--
--
<a href="http://lkcl.net">http://lkcl.net</a>
--