ReactOS 0.3.11 Released

[Dmitry Gorbachev]

Hi!

I see that the release was not announced on ros-general and on
ros-announce lists…

[Ged Murphy]

This is because Aleksey normally does this but he is on holiday at present.
I'll do it later this morning.

Cheers,
Ged

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On Behalf Of Dmitry Gorbachev
Sent: 17 December 2009 00:45
To: ReactOS Development List
Subject: [ros-dev] ReactOS 0.3.11 Released

Hi!

I see that the release was not announced on ros-general and on
ros-announce lists…

_______________________________________________
Ros-dev mailing list
[e-mail]
http://www.reactos.org/mailman/listinfo/ros-dev

[Dmitry Gorbachev]

> This is because Aleksey normally does this but he is on holiday at present.
> I'll do it later this morning.

It seems that there is no announcement on those lists. I am not
subscribed to them, but in archives, there are no new messages.

Dmitry.

[Ged Murphy]

Apologies, I've been as busy as Aleksey recently.
This is now done.

Thanks Dmitry.

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On
Behalf Of Dmitry Gorbachev
Sent: 18 December 2009 23:51
To: ReactOS Development List
Subject: Re: [ros-dev] ReactOS 0.3.11 Released
> This is because Aleksey normally does this but he is on holiday at

present.
> I'll do it later this morning.

It seems that there is no announcement on those lists. I am not
subscribed to them, but in archives, there are no new messages.

Dmitry.

_______________________________________________
Ros-dev mailing list
[e-mail]
http://www.reactos.org/mailman/listinfo/ros-dev

[Jose Catena]

This is very strange, the published source for 0.3.11 causes compile errors
here. I compiled without problems all recent versions from svn up to 44678.
The published source must not be the one used to compile the release, or
something really weird is happening.
I thought you wanted to know.

Jose Catena
DIGIWAVES S.L.

[Sir Gallantmon]

Did you try running "clean" at the RosBE prompt before attempting to build
the code?

On Mon, Dec 21, 2009 at 1:49 AM, Jose Catena [e-mail] wrote:
> This is very strange, the published source for 0.3.11 causes compile errors
> here. I compiled without problems all recent versions from svn up to 44678.
> The published source must not be the one used to compile the release, or
> something really weird is happening.
> I thought you wanted to know.
>
> Jose Catena
> DIGIWAVES S.L.
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

[Jose Catena]

Did you try running "clean" at the RosBE prompt before attempting to build
the code?

Yes, of course. I also tried on a new fresh extracted copy.

Jose Catena

DIGIWAVES S.L.

From: [e-mail] [mailto:[e-mail]] On
Behalf Of Sir Gallantmon
Sent: Monday, 21 December, 2009 13:28
To: ReactOS Development List
Subject: Re: [ros-dev] ReactOS 0.3.11 source broken?

Did you try running "clean" at the RosBE prompt before attempting to build
the code?

On Mon, Dec 21, 2009 at 1:49 AM, Jose Catena [e-mail] wrote:

This is very strange, the published source for 0.3.11 causes compile errors
here. I compiled without problems all recent versions from svn up to 44678.
The published source must not be the one used to compile the release, or
something really weird is happening.
I thought you wanted to know.

Jose Catena
DIGIWAVES S.L.

_______________________________________________
Ros-dev mailing list
[e-mail]
http://www.reactos.org/mailman/listinfo/ros-dev

[Gregor Schneider]

The source for release versions differes slightly from the SVN source.
Still the releases are built from release source, so it should compile with
the appropriate RosBE version.
What type of error did you get?

2009/12/21 Jose Catena [e-mail]
> This is very strange, the published source for 0.3.11 causes compile errors
> here. I compiled without problems all recent versions from svn up to 44678.
> The published source must not be the one used to compile the release, or
> something really weird is happening.
> I thought you wanted to know.
>
> Jose Catena
> DIGIWAVES S.L.
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

[Jose Catena]

Never mind, it was my fault. Now I compiled 0.3.11 sources without stopper
errors.

I don't know what was the cause of the errors, they disappeared after
reinstalling RosBE.

It's curious that the RosBE version is the same (1.4.5), and that previous
install was compiling svn versions correctly, but I don't think it deserves
any further investigation once solved.

Jose Catena

DIGIWAVES S.L.

From: [e-mail] [mailto:[e-mail]] On
Behalf Of Gregor Schneider
Sent: Monday, 21 December, 2009 14:05
To: ReactOS Development List
Subject: Re: [ros-dev] ReactOS 0.3.11 source broken?

The source for release versions differes slightly from the SVN source.
Still the releases are built from release source, so it should compile with
the appropriate RosBE version.
What type of error did you get?

2009/12/21 Jose Catena [e-mail]

This is very strange, the published source for 0.3.11 causes compile errors
here. I compiled without problems all recent versions from svn up to 44678.
The published source must not be the one used to compile the release, or
something really weird is happening.
I thought you wanted to know.

Jose Catena
DIGIWAVES S.L.

_______________________________________________
Ros-dev mailing list
[e-mail]
http://www.reactos.org/mailman/listinfo/ros-dev

[Timo Kreuzer]

I also had problems with RosBE 1.4.5. Probably because it was a
prerelease that I had installed. This wasn't obvious when looking at the
RosBE output. It Only said 1.4.5. I suggest to mark prereleases to help
avoid such problems.

Jose Catena wrote:
> Never mind, it was my fault. Now I compiled 0.3.11 sources without stopper
> errors.
>
> I don't know what was the cause of the errors, they disappeared after
> reinstalling RosBE.
>
> It's curious that the RosBE version is the same (1.4.5), and that previous
> install was compiling svn versions correctly, but I don't think it deserves
> any further investigation once solved.
>
> Jose Catena
>
> DIGIWAVES S.L.
>
> From: [e-mail] [mailto:[e-mail]] On
> Behalf Of Gregor Schneider
> Sent: Monday, 21 December, 2009 14:05
> To: ReactOS Development List
> Subject: Re: [ros-dev] ReactOS 0.3.11 source broken?
>
> The source for release versions differes slightly from the SVN source.
> Still the releases are built from release source, so it should compile with
> the appropriate RosBE version.
> What type of error did you get?
>
> 2009/12/21 Jose Catena [e-mail]
>
> This is very strange, the published source for 0.3.11 causes compile errors
> here. I compiled without problems all recent versions from svn up to 44678.
> The published source must not be the one used to compile the release, or
> something really weird is happening.
> I thought you wanted to know.
>
> Jose Catena
> DIGIWAVES S.L.
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

[Jose Catena]

I know only RosBE is officially supported. I don't have any problem with
that.
But I prefer to work on Visual Studio, and perhaps someone that also intends
to use it may be interested in the following:

Per file custom build steps vs. custom build rules files.
The proper way to add support for external tools like the as and nasm
assemblers, or any other, is to write a .rules file for each of these tools.
This way it is not necessary to specify custom build rules for each file.
The rules file for masm is included in vs8, 9 and 10. Once you load a rules
file into a project, associated extensions are handled just like any other
tool there, making available inheritable properties templates and all that. 
For example, I wrote an as_mscpp.rules file that reads:

<?xml version="1.0" encoding="utf-8"?>
<VisualStudioToolFile
    Name="s as (gnu_as mscpp)"
    Version="8.00"
    >
    <Rules>
    	<CustomBuildRule
    		Name="s_as_mscpp"
    		DisplayName="s (gnu_as mscpp)"
    		CommandLine="cl /E [sIncPaths] [sPPDefs]
$(InputPath) | as -o [sOutF]"
    		Outputs="[$sOutF]"
    		FileExtensions="*.s"
    		ExecutionDescription="Assembling "
    		>
    		<Properties>
    			<StringProperty
    				Name="sOutF"
    				DisplayName="Obj File"
    				Description="Obj File (-o [file])"
    				Switch="&quot;[value]&quot;"
    				Inheritable="true"
    
DefaultValue="$(IntDir)$(InputName).obj"

/>
    			<StringProperty
    				Name="sIncPaths"
    				DisplayName="Inc Paths"
    				Description="Include serach paths
(/I [path])"
    				Switch="/I &quot;[value]&quot;"
    				Delimited="true"
    				Inheritable="true"
    			/>
    			<StringProperty
    				Name="sPPDefs"
    				DisplayName="Preproc Defs"
    				Description="Preprocessor
Definitions (/D [symbol])"
    				Switch="/D &quot;[value]&quot;"
    				Delimited="true"
    				Inheritable="true"
    			/>
    		</Properties>
    	</CustomBuildRule>
    </Rules>
</VisualStudioToolFile>

I have more of these for nasm, wmc, etc.

Also, I don't know if I should submit patches for msvc build, I suppose I
should even if it is not the official way, because I find msvc/gnuc
conditionals there, but the msvc part is broken in many places. I fixed
ntoskrnl and its dependencies issues (many in crt) for msvc. Please let me
know if I should submit such patches or not, and what would be the proper
way to submit these and any other patches. I suppose that initially I should
send them to someone for review, right?

I'd prefer to avoid irc or any instant msg system, I hope this channel is
adequate for any communication regarding development.

I'm very pleased with the improvements in kd and windbg support. I was
working on it and found that I wasted some efforts. No problem, I'll check
svn more frequently for changes, but I'd like to know if there is a list
somewhere with a description of who is currently doing or planning to do
what, to avoid doing the same as someone else again.

I forgot to say, I finished a large project and I'll be dedicating part of
my time to reactos for a while, partial time, perhaps until February. Not
much, because I'll need to earn money again soon, but I hope I could at
least get ntoskrnl running in a regular xp or s2003 install for testing,
fixing any compatibility issues, and then incorporating to it a new high
performance scheduler (I'm very sensitive to the very low performance of
windows in this area). If in the near future I can keep giving some time to
this, I'll take a look at bugs in any area, unimplemented things, or perhaps
I could help a bit with the audio area. I'd love to work with this much
more, as os development has been always my specialty and preferred area, but
I have to earn my living.

Jose Catena
DIGIWAVES S.L.

[Dmitry Gorbachev]

Hi!

Probably you should be granted write access to the repository and you will 
work in a branch.

[Javier Agustìn Fernàndez Arroyo]

isn't KJK::hyperion working on this?

On Mon, Dec 21, 2009 at 12:46 PM, Dmitry Gorbachev
[e-mail]:
> Hi!
>
> Probably you should be granted write access to the repository and you will
> work in a branch.
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

[KJK::Hyperion]

Jose Catena wrote:
> Also, I don't know if I should submit patches for msvc build,

I "mantain" the makefile-based builder, which is the core build backend
AND supports Microsoft compilers (sort of). Ged Murphy maintains the new
Visual Studio project generator (which is based on the makefile-based
builder), while the old project generator is a total orphan
> I suppose I
> should even if it is not the official way, because I find msvc/gnuc
> conditionals there, but the msvc part is broken in many places.

yes, it's supposed to work. Yes, we accept fixes. No, you can't link
with the Microsoft linker when using the makefile-based builder (yet),
because I didn't write the build rules for it (yet). Yes, you will be
able to, eventually, and then use the makefile-based Visual Studio projects
> I fixed
> ntoskrnl and its dependencies issues (many in crt) for msvc. Please let me
> know if I should submit such patches or not, and what would be the proper
> way to submit these and any other patches. I suppose that initially I should
> send them to someone for review, right?

Post them on bugzilla, assign them to me and Cc [e-mail]
> I'm very pleased with the improvements in kd and windbg support. I was
> working on it and found that I wasted some efforts. No problem, I'll check
> svn more frequently for changes, but I'd like to know if there is a list
> somewhere with a description of who is currently doing or planning to do
> what, to avoid doing the same as someone else again.

[e-mail] and [e-mail] are your men. I'd still
recommend using IRC though, as most of the developers hang out there
> and then incorporating to it a new high performance scheduler

Alex Ionescu will wear your spleen like a hat for this. Discuss it with
him first if you want the remotest possibility of your scheduler being
accepted in the tree

[Jose Catena]

> Post them on bugzilla, assign them to me and Cc [e-mail]

Thanks you. I'll post all together after tests completion, including
verification of that I didn't break rosbe/gnuc build in any way.
> [e-mail] and [e-mail] are your men. I'd still
> recommend using IRC though, as most of the developers hang out there

Well, perhaps I'll try IRC sometime, but based in previous experiences, I
don't think it's an efficient communication channel for things like this.
I hope we could manage well enough with these e-mail lists or direct
e-mail with these that you kindly provided.
> Alex Ionescu will wear your spleen like a hat for this. Discuss it with
> him first if you want the remotest possibility of your scheduler being
> accepted in the tree

Hehehe, I won't "discuss" much with him. I'll send to this list an
explanation
of what I intend to do, why, and how. The possibility of overcoming
the real-time scheduling limitations of windows (mostly due to DPC handling,
whose mere existence is one of the effects of an incapable scheduler), is
in my eyes one of the most appealing aspects of reactos. I have been
developing mostly for automation systems and pro audio, and I know well
the problem and how to solve it. If a windows compatible os fixes such
limitations, which is what I intend to do, I can assure you those industries
will be very interested.
In any case, if it is not accepted initially, perhaps at a later time, after
you can see a working implementation, much simpler than current one,
yet much more powerful and efficient. But if it's still not accepted,
I'm still willing to do it privately and I hope in such a case you won't
have
any problem with me using reactos sources for that.

Best regards and thanks you very much for answering my questions.

Jose Catena
DIGIWAVES S.L.

[Olaf Siejka]

As mail-list might be cumbersome in some situations, please consider joining
us on irc.freenode.net channel #reactos as you can ask questions and get
your answers swifter, with a bit of luck.

best regards

2009/12/21 Jose Catena [e-mail]
> Best regards and thanks you very much for answering my questions.
>
> Jose Catena
> DIGIWAVES S.L.
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

[Alex Ionescu]

If your new implementation:

1) Is better than what Windows does today (hint: it's nearly lockless in Win 7, and O(1) since 2003) in every single way (ie: not sacrificing 50% of desktop users for 10% of server users).

AND

2) Maintains full compatibility with Windows applications (and I expect you to TEST this), drivers, etc in every way.

I promise you I will wholeheartedly support its inclusion in ReactOS.

In fact, I will do even more than just that.

On 2009-12-21, at 5:53 PM, Jose Catena wrote:
> > Post them on bugzilla, assign them to me and Cc [e-mail]
>
> Thanks you. I'll post all together after tests completion, including
> verification of that I didn't break rosbe/gnuc build in any way.
> > [e-mail] and [e-mail] are your men. I'd still
> > recommend using IRC though, as most of the developers hang out there
>
> Well, perhaps I'll try IRC sometime, but based in previous experiences, I
> don't think it's an efficient communication channel for things like this.
> I hope we could manage well enough with these e-mail lists or direct
> e-mail with these that you kindly provided.
> > Alex Ionescu will wear your spleen like a hat for this. Discuss it with
> > him first if you want the remotest possibility of your scheduler being
> > accepted in the tree
>
> Hehehe, I won't "discuss" much with him. I'll send to this list an
> explanation
> of what I intend to do, why, and how. The possibility of overcoming
> the real-time scheduling limitations of windows (mostly due to DPC handling,
> whose mere existence is one of the effects of an incapable scheduler), is
> in my eyes one of the most appealing aspects of reactos. I have been
> developing mostly for automation systems and pro audio, and I know well
> the problem and how to solve it. If a windows compatible os fixes such
> limitations, which is what I intend to do, I can assure you those industries
> will be very interested.
> In any case, if it is not accepted initially, perhaps at a later time, after
> you can see a working implementation, much simpler than current one,
> yet much more powerful and efficient. But if it's still not accepted,
> I'm still willing to do it privately and I hope in such a case you won't
> have
> any problem with me using reactos sources for that.
>
> Best regards and thanks you very much for answering my questions.
>
> Jose Catena
> DIGIWAVES S.L.
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

Best regards,
Alex Ionescu

[Jose Catena]

I'm very glad you hear you saying that, Alex.
Those are indeed requirements my implementation will have to comply with,
and I think it is totally feasible.
It should improve realtime class latency very meaningfully without making
any other parameter worse, although I also expect to achieve improvements in
other parameters too.
What I intend to do is to fully comply with the real time paradigm, by using
some very efficient solutions I have been using to eliminate the risks of rt
priorities misuses without breaking the paradigm. For me that's the easy
thing (and proven already), although some related ppl at MS had a very hard
time to understand it.
The real challenge is to handle DPCs as preemptable real time threads, fully
respecting assigned rt priorities. This is what would fix the problem
definitely, but since currently DPCs are not preemtable, there is a
possibility of breaking some drivers because access serialization or sync
issues. But I expect this possibility will be in real world very low, as
there should not be interactions between DPCs created by different drivers
out of system control, and ultimately, the new scheduler will be very
flexible, allowing to configure each driver's DPCs parameters separately, so
DPC preemption could be disabled for each particular driver or for all of
them. Probably the default config will be fully compatible with current
behavior, but I expect that enabling DPC preemption by lowering the
priorities of selected driver's DPCs should work fine with the most or
almost all drivers.
I could post a more concise description of the plan, but what I intend
ultimately to achieve is a fully working and tested ntoskrnl that could run
on regular xp or s2003 too (to verify full compatibility). So maybe instead
of writing the details and discussing it, we may wait till I have it working
and speak again afterwards. This way I won't be wasting the time of anyone
else until the objectives are achieved and can be verified.

P.D: Win7 has again improved the scheduler in the right direction (and fixed
a related big mistake in the Vista's WaveRT model), but the DPC thing still
kills the latency and they don't want to change that. I already discussed it
with some key ppl at MS: a few understood it, but even those didn't think
there were good chances of convincing decision makers anytime soon. It is
considered a very low priority, if not null at all. But for all pro audio
manufacturers, and some other niches like automation and control, it is a
top priority.

Best regards,

Jose Catena
DIGIWAVES S.L.

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On
Behalf Of Alex Ionescu
Sent: Tuesday, 22 December, 2009 04:10
To: ReactOS Development List
Subject: Re: [ros-dev] MSVC

If your new implementation:

1) Is better than what Windows does today (hint: it's nearly lockless in Win
7, and O(1) since 2003) in every single way (ie: not sacrificing 50% of
desktop users for 10% of server users).

AND

2) Maintains full compatibility with Windows applications (and I expect you
to TEST this), drivers, etc in every way.

I promise you I will wholeheartedly support its inclusion in ReactOS.

In fact, I will do even more than just that.

[Alex Ionescu]

Sounds like you want make Threaded DPCs (which exist to fill the niche you talked about) the default model. Are you aware of Threaded DPCs? Why not just go down that route?

Your idea of targeting DPC-heavy drivers to lower their throughput is what MMCSS attempted (and still does) to do back in Vista -- it partly lead to a catastrophe as suddenly network card traffic on heavy audio I/O machines dropped to a mere fraction of full network bandwidth on a GigE network. What techniques have you developed to address these issues?

Other DPC throttling on I/O leads to priority inversion and inheritance problems, due to the heavy inter-dependency of Windows services and drivers. How do you plan on solving these challenges?

Most scalability bottlenecks in the 2003 scheduler are related to the dispatcher lock -- multiple highly threaded real time applications would encounter increased latency on many-core systems. How do you intend to solve this issue without removing the lock as well (which would cause multiple layer/version violations in ReactOS and take nearly as much time as removing it in Win7 took)?

How do you intend to balance real-time thread usage with multiple user/TS scenarios?

Also, I'm not sure what WaveRT/scheduler mode you are talking about. WaveRT was mostly changed to properly support Event-Mode in Win7, which is an issue entirely separate to scheduler changes.

Which "key" people have you spoken with? Have you spoken with Arun Kishan?

On 2009-12-21, at 11:40 PM, Jose Catena wrote:
> I'm very glad you hear you saying that, Alex.
> Those are indeed requirements my implementation will have to comply with,
> and I think it is totally feasible.
> It should improve realtime class latency very meaningfully without making
> any other parameter worse, although I also expect to achieve improvements in
> other parameters too.
> What I intend to do is to fully comply with the real time paradigm, by using
> some very efficient solutions I have been using to eliminate the risks of rt
> priorities misuses without breaking the paradigm. For me that's the easy
> thing (and proven already), although some related ppl at MS had a very hard
> time to understand it.
> The real challenge is to handle DPCs as preemptable real time threads, fully
> respecting assigned rt priorities. This is what would fix the problem
> definitely, but since currently DPCs are not preemtable, there is a
> possibility of breaking some drivers because access serialization or sync
> issues. But I expect this possibility will be in real world very low, as
> there should not be interactions between DPCs created by different drivers
> out of system control, and ultimately, the new scheduler will be very
> flexible, allowing to configure each driver's DPCs parameters separately, so
> DPC preemption could be disabled for each particular driver or for all of
> them. Probably the default config will be fully compatible with current
> behavior, but I expect that enabling DPC preemption by lowering the
> priorities of selected driver's DPCs should work fine with the most or
> almost all drivers.
> I could post a more concise description of the plan, but what I intend
> ultimately to achieve is a fully working and tested ntoskrnl that could run
> on regular xp or s2003 too (to verify full compatibility). So maybe instead
> of writing the details and discussing it, we may wait till I have it working
> and speak again afterwards. This way I won't be wasting the time of anyone
> else until the objectives are achieved and can be verified.
>
> P.D: Win7 has again improved the scheduler in the right direction (and fixed
> a related big mistake in the Vista's WaveRT model), but the DPC thing still
> kills the latency and they don't want to change that. I already discussed it
> with some key ppl at MS: a few understood it, but even those didn't think
> there were good chances of convincing decision makers anytime soon. It is
> considered a very low priority, if not null at all. But for all pro audio
> manufacturers, and some other niches like automation and control, it is a
> top priority.
>
> Best regards,
>
> Jose Catena
> DIGIWAVES S.L.
>
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On
> Behalf Of Alex Ionescu
> Sent: Tuesday, 22 December, 2009 04:10
> To: ReactOS Development List
> Subject: Re: [ros-dev] MSVC
>
> If your new implementation:
> Is better than what Windows does today (hint: it's nearly lockless in Win
>
> 7, and O(1) since 2003) in every single way (ie: not sacrificing 50% of
> desktop users for 10% of server users).
>
> AND
> Maintains full compatibility with Windows applications (and I expect you
>
> to TEST this), drivers, etc in every way.
>
> I promise you I will wholeheartedly support its inclusion in ReactOS.
>
> In fact, I will do even more than just that.
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

Best regards,
Alex Ionescu

[Jose Catena]

As intended in my last post, better work and proof than discussion. Do you
agree?
You made too many assumptions, forget for a while that you already know
everything if you want to learn something.
What an audio workstation needs is that the audio events don't delay beyond
a predictable maximum. The processing of such events will be always a
fraction of the delay. So if the maximum latency is for example 1ms, lower
priority rt tasks won't be delayed more than a fraction of a millisecond for
this cause. This does not necessarily affect throughput of anything, and
anyway never without a good reason. An audio workstation will configure the
audio driver to have higher priority DPCs than disk, and disk higher than
network, USB, video... A network server will give priority to NIC then to
disk then whatever. Is the current scheme better, when all NIC, audio
driver, disk will suffer because the mouse DPCs? Proper RT scheduling
without anything violating it (like DPCs) can only help in every scenario,
how couldn't it? How could the current scheme be more efficient in latency,
throughput or anything than this one? Are you joking or what?
The rt priorities don't have much to do with throughput, the tasks demanding
lowest latency are the ones that will also do its job very quickly. If a rt
task does not finish in a very limited time, it does not comply and its
priority will be changed to time sharing range, this is one of the keys in
my scheme. The useful statistics are very easy to gather efficiently.
The threaded DPCs are not a solution when it depends on all drivers
employing it (which is not a reality and will not be), and does not address
the need of configuration of the priorities scheme by the user. Nor Win7
solves the risks of user configurable real time priorities.
MMCSS is not that thing, it is just a try to provide a reserved high
priority class to user mode, but it is anyway below *any* and *all* DPCs.
What happened with WaveRT in Vista is a demonstration of the absolute lack
of knowledge of what realtime scheduling is and the effects of latency, and
being separate of the scheduler itself, is a proof of the lack of
understanding in the very same regard. And very related to the scheduler
too. If it was so hard to learn for them after endless discussions, I
wouldn't expect you would a more complex timings relationship, much less
knowing your "I know it all" attitude.
You make me laugh, so if my scheduler performs better than the one in Win7
(not only better than the ones in ReactOS) you will support its inclusion?
LOL.
My aim is well beyond Win7 performance anyway. I'll work on it, and if I
success as I expect, you will support its inclusion or not as you wish. But
please don't ask me to discuss in that tone and attitude, we all know you're
the smartest guy with the biggest dick and I won't waste time challenging
you, ok?
I don't know Arun Kishan, who is he? MS is so huge... Most ppl I treated is
in the wdm area. Also as a team of audio developers leaded by Ron Kuper
(Cakewalk) we have been working with them at all levels. After a decade or
so the improvements are valuable, we got the KS interfaces published, the
nasty priority inversion algorithm removed, an so much more. And still, so
far from the right thing. If I did alone a scheduler that solved elegantly
and efficiently the same problems (and mine is not as large as yours by
far), how a company with the resources of MS doesn't? It is not that they
don't have great people, it is not that they don't get well founded and
documented requests, it is not a black hand... is the very nature of
marketing, every user will find some icons nicer than others, but hardly
understands what low latency means. If not, look how you saw at it: if MS
didn't how you...? If a developer skilled in windows internals doesn't know
about real time concepts...
Shields down. It is fun but I don't want to waste time in useless
discussions. I'll keep working as I said, and you all will know how it goes.
The only big obstacle in the way is the very limited time I have for it, not
any of arguments. Not that changing the DPC handling and sync will be easy,
but the model I'm trying to implement works great, I have already used it in
very different projects. This is not a fool idea of the latest frickie.
Back to msvc build fixing...
Cheers,

Jose Catena
DIGIWAVES S.L.

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On
Behalf Of Alex Ionescu
Sent: Tuesday, 22 December, 2009 06:36
To: ReactOS Development List
Subject: Re: [ros-dev] MSVC

Sounds like you want make Threaded DPCs (which exist to fill the niche you
talked about) the default model. Are you aware of Threaded DPCs? Why not
just go down that route?

Your idea of targeting DPC-heavy drivers to lower their throughput is what
MMCSS attempted (and still does) to do back in Vista -- it partly lead to a
catastrophe as suddenly network card traffic on heavy audio I/O machines
dropped to a mere fraction of full network bandwidth on a GigE network. What
techniques have you developed to address these issues?

Other DPC throttling on I/O leads to priority inversion and inheritance
problems, due to the heavy inter-dependency of Windows services and drivers.
How do you plan on solving these challenges?

Most scalability bottlenecks in the 2003 scheduler are related to the
dispatcher lock -- multiple highly threaded real time applications would
encounter increased latency on many-core systems. How do you intend to solve
this issue without removing the lock as well (which would cause multiple
layer/version violations in ReactOS and take nearly as much time as removing
it in Win7 took)?

How do you intend to balance real-time thread usage with multiple user/TS
scenarios?

Also, I'm not sure what WaveRT/scheduler mode you are talking about. WaveRT
was mostly changed to properly support Event-Mode in Win7, which is an issue
entirely separate to scheduler changes.

Which "key" people have you spoken with? Have you spoken with Arun Kishan?

[Ged Murphy]

Jose Catena wrote:
> If it was so hard to learn for them after endless discussions, I
> wouldn't expect you would a more complex timings relationship, much less
> knowing your "I know it all" attitude.
> You make me laugh, so if my scheduler performs better than the one in Win7
> (not only better than the ones in ReactOS) you will support its inclusion?
> LOL.
> My aim is well beyond Win7 performance anyway. I'll work on it, and if I
> success as I expect, you will support its inclusion or not as you wish.

But
> please don't ask me to discuss in that tone and attitude, we all know

you're
> the smartest guy with the biggest dick and I won't waste time challenging
> you, ok?

Can I ask which part of Alex's email you found offensive?
I'm confused as to why you felt the need be rude and offensive.

Please don't turn our mailing list into a place to pick arguments, this is
what IRC is for.

Ged.

[KJK::Hyperion]

Jose Catena wrote:
> I could post a more concise description of the plan, but what I intend
> ultimately to achieve is a fully working and tested ntoskrnl that could run
> on regular xp or s2003 too (to verify full compatibility).

As a general rule, we prefer fixes to features, and even then we prefer
features found in Windows to entirely new features. As an example of why
we try to avoid new features: running DPCs in pre-emptible real-time
threads raises serious issues, like the risk of deadlocks between high
priority real-time applications and DPCs that used to run at a higher
priority but don't anymore. You'll need a framework for compatibility
settings to make it work with existing drivers: at that point you'll
have solved one problem and created two new ones

If you want to make a lasting contribution to the project, work on
something that's aligned to the goals of the project
> So maybe instead
> of writing the details and discussing it, we may wait till I have it working
> and speak again afterwards. This way I won't be wasting the time of anyone
> else until the objectives are achieved and can be verified.

One doesn't usually start a discussion about something and at the same
time dismiss the need to discuss it. If you came here to brag, you came
to the wrong place

[Alexandru Lovin]

> One doesn't usually start a discussion about something and at the same
> time dismiss the need to discuss it. If you came here to brag, you came
> to the wrong place

May I kindly suggest that he came here to ask if you are interested in what
he wants to do ? Just to know if he has the support of the team. That's what
it looks like from a non-programmer's point of view.

[Jose Catena]

Ø  May I kindly suggest that he came here to ask if you are interested in
what he wants to do ? Just to know if he has the support of the team. That's
what it looks like from a non-programmer's point of view.

1)      I was simply asking if I should submit fixes and how.

2)      I was telling what I’m doing for your information. I didn’t ask for
integration of my project in ReactOS, I’ll only ask for it if I finish it
successfully.

3)      No, I’m not asking for support in that sense, I intend to write the
new scheduler alone. I don’t ignore that it will take a lot of work, but I
couldn’t expect any involvement by the ReactOS team.

4)       I didn’t ask for discussion or help. I’m well aware of the
difficulties and solutions, well beyond the well intentioned objections
raised.

I hope this clears any confusion and ends any discussion on the subject.

Jose Catena

DIGIWAVES S.L.

[KJK::Hyperion]

Jose Catena wrote:
> well beyond the well intentioned objections raised.

Sorry, the position of "official project asshole" is already taken. Try
maybe applying for "respectful newbie" instead

[Jose Catena]

> If you want to make a lasting contribution to the project, work on
> something that's aligned to the goals of the project

I agree with you fully in that fixing existing features and replicating
those on windows is better for ReactOS.
But I have a strong personal interest in this issue, and this is where I'll
be working for now.
Regardless of if ReactOS at the end uses the new scheduler or not, I will be
fixing any bugs and compatibility issues in ntoskrnl that I find in the way,
so there will be direct benefits for ReactOS anyway.
Also, I'm finding a lot of chaos with the crt, ndk, psdk, ddk, and mingw
includes. I'm trying to leave that as is to keep sync with ROS and because
fixing those would be a lot of work, but as I work more I see the need to
fix many. Perhaps I'll have to engage a major review of all that despite its
cost, which would be very good for ReactOS as well.
And as I said, afterwards I intend to work on general bug and compatibility
fixing... subject to time availability.
> One doesn't usually start a discussion about something and at the same
> time dismiss the need to discuss it. If you came here to brag, you came
> to the wrong place

I stated I didn't want to discuss, and this ends here. I only wanted to tell
you what I'm doing and asking how should I commit fixes.
I'm sorry for my rude answer to Alex. I tend to be rude, yes. I hate useless
discussions. I should not have answered that way.

Cheers,

Jose Catena
DIGIWAVES S.L.

[Jose Catena]

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On
Behalf Of KJK::Hyperion
> Post them on bugzilla, assign them to me and Cc [e-mail]

Well, I submitted my first bug & patch to bugzilla.
Before I submit more, I'd like to know if I did it correctly. I have many
and I'll wait to know if should submit them this way.
Couldn't cc to [e-mail], bugzilla doesn't know him.
Based on the large amount and severity of bugs in the msvc build perhaps
nobody is using it but me, plz let me know if you don't want any fixes to
msvc build submitted.

The submitted bug & patch is as follows:

Bug# 5071 intrin_i.h: sgdt & lgdt fixed for msvc

In intrin_i.h there are two inline functions that are defined differently
based on __GNUC__ or _MSV_VER. The _MSC_VER ones make ntoskrnl crash early
on boot.
As these functions were written, sgd & lgdt stored and loaded the gdt
to/from the pointer variable passed instead of the location the pointer
points to.
Fixed and tested.
Patch follows:

intrin_i.h

Ke386GetGlobalDescriptorTable(OUT PVOID Descriptor)
{
-	__asm sgdt [Descriptor]
+	__asm {
+		mov ebx, Descriptor
+		sgdt [ebx];
+	}
}

Ke386SetGlobalDescriptorTable(IN PVOID Descriptor)
{
-	__asm lgdt [Descriptor]
+	__asm {
+		mov ebx, Descriptor;
+		lgdt [ebx];
+	}
}

Jose Catena
DIGIWAVES S.L.

[Alex Ionescu]

I recommend changing the convention such that Descriptor is a pointer to the pointer -- this way the functions can remain one-liners and not introduce register side-effects (especially since you're choosing ebx -- trashing a nonvolatile).

On 2009-12-30, at 11:46 AM, Jose Catena wrote:
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On
> Behalf Of KJK::Hyperion
> > Post them on bugzilla, assign them to me and Cc [e-mail]
>
> Well, I submitted my first bug & patch to bugzilla.
> Before I submit more, I'd like to know if I did it correctly. I have many
> and I'll wait to know if should submit them this way.
> Couldn't cc to [e-mail], bugzilla doesn't know him.
> Based on the large amount and severity of bugs in the msvc build perhaps
> nobody is using it but me, plz let me know if you don't want any fixes to
> msvc build submitted.
>
> The submitted bug & patch is as follows:
>
> Bug# 5071 intrin_i.h: sgdt & lgdt fixed for msvc
>
> In intrin_i.h there are two inline functions that are defined differently
> based on __GNUC__ or _MSV_VER. The _MSC_VER ones make ntoskrnl crash early
> on boot.
> As these functions were written, sgd & lgdt stored and loaded the gdt
> to/from the pointer variable passed instead of the location the pointer
> points to.
> Fixed and tested.
> Patch follows:
>
> intrin_i.h
>
> Ke386GetGlobalDescriptorTable(OUT PVOID Descriptor)
> {
> __asm sgdt [Descriptor]
> __asm {
> mov ebx, Descriptor
> sgdt [ebx];
> }
>
> }
>
> Ke386SetGlobalDescriptorTable(IN PVOID Descriptor)
> {
> __asm lgdt [Descriptor]
> __asm {
> mov ebx, Descriptor;
> lgdt [ebx];
> }
>
> }
>
> Jose Catena
> DIGIWAVES S.L.
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

Best regards,
Alex Ionescu

[Jose Catena]

eax then.
A pointer to the pointer won't help, only make the problem worse needing an
additional indirection.
AFIK, there is no way to tell the inline assembler how to do it correctly in
a single instruction, we need to have the pointer loaded in a register. A
fastcall will have the parameter in a register already, but won't be better
than an inline.
There is no point in saving a instruction here anyway, loading/saving the
gdtr is very rare.
But if you know a way to have a single instruction in an inline to work as
intended tell us how plz.

Jose Catena
DIGIWAVES S.L.

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On
Behalf Of Alex Ionescu
Sent: Wednesday, 30 December, 2009 18:08
To: ReactOS Development List
Subject: Re: [ros-dev] MSVC

I recommend changing the convention such that Descriptor is a pointer to the
pointer -- this way the functions can remain one-liners and not introduce
register side-effects (especially since you're choosing ebx -- trashing a
nonvolatile).

[Alex Ionescu]

Sorry, I meant pass it byval with a macro.

On 2009-12-30, at 12:49 PM, Jose Catena wrote:
> eax then.
> A pointer to the pointer won't help, only make the problem worse needing an
> additional indirection.
> AFIK, there is no way to tell the inline assembler how to do it correctly in
> a single instruction, we need to have the pointer loaded in a register. A
> fastcall will have the parameter in a register already, but won't be better
> than an inline.
> There is no point in saving a instruction here anyway, loading/saving the
> gdtr is very rare.
> But if you know a way to have a single instruction in an inline to work as
> intended tell us how plz.
>
> Jose Catena
> DIGIWAVES S.L.
>
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On
> Behalf Of Alex Ionescu
> Sent: Wednesday, 30 December, 2009 18:08
> To: ReactOS Development List
> Subject: Re: [ros-dev] MSVC
>
> I recommend changing the convention such that Descriptor is a pointer to the
> pointer -- this way the functions can remain one-liners and not introduce
> register side-effects (especially since you're choosing ebx -- trashing a
> nonvolatile).
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

Best regards,
Alex Ionescu

[Jose Catena]

The kernel built with _WINKD_ = 1 and KDBG = 0 crashes early here.
Since it is reported as working I'd like to know if there is some trick I
don't know before I investigate further.
I assume only notoskrnl and kdcom.dll are different.
I also tried using the kdcom.dll from w2003 without any difference.
Anyone that tried successfully windbg plz tell me how.

Jose Catena
DIGIWAVES S.L.

[Daniel Reimer]

It does not crash. It waits for a connection to windbg and continues as 
soon as its established.

Jose Catena schrieb:
> The kernel built with _WINKD_ = 1 and KDBG = 0 crashes early here.
> Since it is reported as working I'd like to know if there is some trick I
> don't know before I investigate further.
> I assume only notoskrnl and kdcom.dll are different.
> I also tried using the kdcom.dll from w2003 without any difference.
> Anyone that tried successfully windbg plz tell me how.
>
> Jose Catena
> DIGIWAVES S.L.
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

[Jose Catena]

> It does not crash. It waits for a connection to windbg and continues as 
> soon as its established.

Thanks you. It does not work here:
I run ros in vmware 6.5 with serial port mapped to pipe. I have used windbg
with xp in the vm (it should mean I have configured the connection
correctly), but the connection with ros is not established.
Furthermore, if I start ros without the debug option, it still halts. Does
it always expect a windbg connection even if no debug option is specified at
boot?
I must be doing something wrong, but I don't know what. At KDBG must be 0 or
1? I compiled with _WINKD_=1 KDBG=0 GDB=0.
Thanks for any ideas.

Jose Catena
DIGIWAVES S.L.

-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On
Behalf Of Daniel Reimer
Sent: Wednesday, 30 December, 2009 20:31
To: ReactOS Development List
Subject: Re: [ros-dev] _WINKD_

[Ged Murphy]

I had similar problems the last time I tried it using the same setup as you.

On Wed, Dec 30, 2009 at 7:49 PM, Jose Catena [e-mail] wrote:
> > It does not crash. It waits for a connection to windbg and continues as
> > soon as its established.
>
> Thanks you. It does not work here:
> I run ros in vmware 6.5 with serial port mapped to pipe. I have used windbg
> with xp in the vm (it should mean I have configured the connection
> correctly), but the connection with ros is not established.
> Furthermore, if I start ros without the debug option, it still halts. Does
> it always expect a windbg connection even if no debug option is specified
> at
> boot?
> I must be doing something wrong, but I don't know what. At KDBG must be 0
> or
> 1? I compiled with _WINKD_=1 KDBG=0 GDB=0.
> Thanks for any ideas.
>
> Jose Catena
> DIGIWAVES S.L.
>
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On
> Behalf Of Daniel Reimer
> Sent: Wednesday, 30 December, 2009 20:31
> To: ReactOS Development List
> Subject: Re: [ros-dev] _WINKD_
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev

[Olaf Siejka]

Could you describe the ROS behaviour more precisely? Are you sure you set up
both parties correctly? I`ll try to check winkd build around tomorow.

On Wed, Dec 30, 2009 at 7:49 PM, Jose Catena [e-mail] wrote:
> > It does not crash. It waits for a connection to windbg and continues as
> > soon as its established.
>
> Thanks you. It does not work here:
> I run ros in vmware 6.5 with serial port mapped to pipe. I have used windbg
> with xp in the vm (it should mean I have configured the connection
> correctly), but the connection with ros is not established.
> Furthermore, if I start ros without the debug option, it still halts. Does
> it always expect a windbg connection even if no debug option is specified
> at
> boot?
> I must be doing something wrong, but I don't know what. At KDBG must be 0
> or
> 1? I compiled with _WINKD_=1 KDBG=0 GDB=0.
> Thanks for any ideas.
>
> Jose Catena
> DIGIWAVES S.L.
>
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On
> Behalf Of Daniel Reimer
> Sent: Wednesday, 30 December, 2009 20:31
> To: ReactOS Development List
> Subject: Re: [ros-dev] _WINKD_
>
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev
> Ros-dev mailing list
> [e-mail]
> http://www.reactos.org/mailman/listinfo/ros-dev