Brainstorming about testing team, roadmap and ReactOS usability

[Aleksey Bragin]

	Hi,
I had a chat with Olaf a few days ago, regarding a few issues, and we  
thought it would be beneficial to bring them up for collective  
brainstorming.

Background information: In the past and nowadays work in ReactOS is  
getting done "as it flows". E.g. if we get developers, who wanted to  
implement sound support, would do it, and we advertise this in a  
release. Or, like Evgeniy Boltik who shows a nice cooperation with  
Timo, Jim and Ged on fixing various nasty palette and drawing issues  
which were not touched for a few years, often fixing errors  
introduced in revisions like 1000 or 4000.
There is no central plan, just a very rough roadmap, mainly  
relatively short-term.

Same applies to testing team: we usually perform testing when an  
obvious problem occurs, in our very well known apps list, which  
someone remembers to work in a certain revision. If that's known to  
regress, Olaf usually organizes a regression-testing, a guilty  
revision is found, and problem hopefully gets fixed.

All of this is very good, for an operating system which is developed  
just for the sake of development: noone sets any milestones, noone  
demands to meet them. And noone pays for achieving them either.

But when Linux got a real spin-off? An answer is: when it aimed to  
become useful for the most needed application - as a platform to run  
a web server and a mail server. http://en.wikipedia.org/wiki/ 
Linux_adoption says that in 2001 Linux servers experienced 15% annual  
growth rate, and by 2004 50% of the worldwide blade servers are  
shipped with Linux-based OS.

This was preceded by hard work of a number of people, like Andrew  
Morton. Who aimed not just at gettign more and more features, but  
more at getting existing code as stable as it's possible. That  
included going through all the issues people reported, asking them to  
provide more info if needed, finding the actual bug, fixing it,  
asking them to retest... At some point all most frequently  
encountered bugs were fixed, and potential Linux usage base extended  
substantially.


For ReactOS, we have different aims, but we could use this strategy  
now, when core parts of our operating system are becoming feature- 
complete and mostly need debugging, and when we finally are getting  
very close to going Beta.

This involves, first of all, getting hardware support at a decent  
level. From a billion of personal computers in the world, we could  
easily support the most common hardware without need to develop  
complex drivers or different HALs. Even SATA would not really be a  
strong requirement, since most of PCs have IDE compatibility mode,  
which many non-technical people even forget to switch to "SATA" mode  
in the BIOS when installing their Windows OS! Again Linux's  
philisophy could help us: they are proud that their hardware is Linux- 
compatible, while we are embarrased that "our OS doesn't support  
specific hardware". Feel the difference.

Some good step in this direction is this wikipage: http:// 
www.reactos.org/wiki/index.php/Supported_Hardware/Network_cards

Next, software. We have a unique situation when our OS targets a vast  
amount of WinAPI compatible software. There is a lot of bugs in our  
Win32 subsystem, however if we can find a set of apps we support  
right now, or find common software which we could easily support - it  
should be our "road ahead".

Now back to what I started writing this email about. I was speaking  
with Olaf and Klemens about the software compatibility database.  
Existing "Support  Database" is not used by people, and not liked by  
most of our developers and testers because it takes more time to file  
an entry into that database than to actually install ReactOS and test  
the app. Another problem is that community-driven database won't work  
since we can't confirm people's data.
What we thought would work good so far is a list of working apps. As  
Olaf said: "it'll be ugly, not scientifically statistically fine,  
crude comparing to wine appdb and limited. Bit it can be done". The  
amount of text to enter for an app entry must be as minimal as  
possible: app name, platform (real hardware, emulator), bug number in  
bugzilla for possible issues (max. 2 bugs for one entry).
Also, no division for "installs, but doesn't start" or "works only  
via manual installation". Either it works, or it doesn't, no  
intermediate stages.
As we thought later, it may even be just three columns: appname, ROS  
revision, Comments.

A wikipage could suit this need, but if necessary, we could think  
further of some web-app.


I think that's enough for one email, I'll be glad to listen to  
comments and more ideas. Everyone is welcome to take part in the  
discussion.

With the best regards,
Aleksey Bragin.

[Steven Edwards]

On Wed, Apr 8, 2009 at 6:06 AM, Aleksey Bragin <aleksey(a)reactos.org> wrote:
>   Background information: In the past and nowadays work
> in ReactOS is
>  getting done "as it flows". E.g. if we get developers, who wanted to
>  implement sound support, would do it, and we advertise this in a
>  release. Or, like Evgeniy Boltik who shows a nice cooperation with
>  Timo, Jim and Ged on fixing various nasty palette and drawing issues
>  which were not touched for a few years, often fixing errors
>  introduced in revisions like 1000 or 4000.
>  There is no central plan, just a very rough roadmap, mainly
>  relatively short-term. 
Everyone is doing a great job nailing down these issues!

<snip>

>   This involves, first of all, getting hardware support
> at a decent
>  level. From a billion of personal computers in the world, we could
>  easily support the most common hardware without need to develop
>  complex drivers or different HALs. Even SATA would not really be a
>  strong requirement, since most of PCs have IDE compatibility mode,
>  which many non-technical people even forget to switch to "SATA" mode
>  in the BIOS when installing their Windows OS! Again Linux's
>  philisophy could help us: they are proud that their hardware is Linux-
>  compatible, while we are embarrased that "our OS doesn't support
>  specific hardware". Feel the difference. 
Yes its a mental shift, but a hard one to adopt. ReactOS having more
value over other operating systems is the selling point. Linux has
been able to show its value over Windows in certain situations, and
that value is greater than the hardware support value. An example,
Google runs Linux, Google could not have built its network running
Windows due to licensing costs. Therefore Linux has more value than
Windows in that situation. It makes it worth the effort to find
hardware that is Linux compatible.

ReactOS related comments at the end....

<snip>
>   What we thought would work good so far is a list of
> working apps. As
>  Olaf said: "it'll be ugly, not scientifically statistically fine,
>  crude comparing to wine appdb and limited. Bit it can be done". The
>  amount of text to enter for an app entry must be as minimal as
>  possible: app name, platform (real hardware, emulator), bug number in
>  bugzilla for possible issues (max. 2 bugs for one entry).
>  Also, no division for "installs, but doesn't start" or "works only
>  via manual installation". Either it works, or it doesn't, no
>  intermediate stages.
>  As we thought later, it may even be just three columns: appname, ROS
>  revision, Comments. 
I agree with this. All software has bugs. I think the (Works/Does Not
Work) is a point of view, of 'can I use this software even with its
bugs?'. If yes, then it works. If not, then it does not work.

Now back to the value thing. Windows costs money, ReactOS can now run
hardware GL and Wine DirectX in select virtual machines. If you want
to show the value of ReactOS its time to make ReactOS run as many
Windows games as possible in these virtual machines on Apple Hardware.
This will save OS X users the need to dual boot or acquire a Windows
license. Also hardware support is not a big problem, which again helps
us. This is the future.

-- 
Steven Edwards

"There is one thing stronger than all the armies in the world, and
that is an idea whose time has come." - Victor Hugo

[Timo Kreuzer]

One of the current issues that makes ReactOS completely useless for real
stuff is not that it cannot run app X or game Y or doesn't support my
network adapter. The most annoying thing is it's instability.
You have no chance to use any of the great features, because most likely
reactos crashes before you are at that point.
Try downloading something that is > 300MB, or installing a bigger
application... good luck :(
It just crashes way too often. We all know our crappy Cc and yes, I know
win32k also crashes from time to time.
And after a crash it's quite likely that things don't work anymore  (for
example downloader) and you need to reinstall reactos. If we could fix
that, the system would look much better than it does today.
Before a 0.4, we should start a stability offensive.
It would also improve developing, because the crashes can really be a
pain when it comes to testing something. Excessive testing is more or
less impossible, because of all the crashes.

Just my 2 cents,
Timo


Steven Edwards schrieb:
>   On Wed, Apr 8, 2009 at 6:06 AM, Aleksey Bragin
> <aleksey(a)reactos.org> wrote:
>    
> >   Background information: In the past and nowadays
> > work in ReactOS is
> >  getting done "as it flows". E.g. if we get developers, who wanted to
> >  implement sound support, would do it, and we advertise this in a
> >  release. Or, like Evgeniy Boltik who shows a nice cooperation with
> >  Timo, Jim and Ged on fixing various nasty palette and drawing issues
> >  which were not touched for a few years, often fixing errors
> >  introduced in revisions like 1000 or 4000.
> >  There is no central plan, just a very rough roadmap, mainly
> >  relatively short-term.
> >       
>  Everyone is doing a great job nailing down these issues!
>
>  <snip>
>
>    
> >   This involves, first of all, getting hardware
> > support at a decent
> >  level. From a billion of personal computers in the world, we could
> >  easily support the most common hardware without need to develop
> >  complex drivers or different HALs. Even SATA would not really be a
> >  strong requirement, since most of PCs have IDE compatibility mode,
> >  which many non-technical people even forget to switch to "SATA" mode
> >  in the BIOS when installing their Windows OS! Again Linux's
> >  philisophy could help us: they are proud that their hardware is Linux-
> >  compatible, while we are embarrased that "our OS doesn't support
> >  specific hardware". Feel the difference.
> >       
>  Yes its a mental shift, but a hard one to adopt. ReactOS having more
>  value over other operating systems is the selling point. Linux has
>  been able to show its value over Windows in certain situations, and
>  that value is greater than the hardware support value. An example,
>  Google runs Linux, Google could not have built its network running
>  Windows due to licensing costs. Therefore Linux has more value than
>  Windows in that situation. It makes it worth the effort to find
>  hardware that is Linux compatible.
>
>  ReactOS related comments at the end....
>
>  <snip>
>    
> >   What we thought would work good so far is a list
> > of working apps. As
> >  Olaf said: "it'll be ugly, not scientifically statistically fine,
> >  crude comparing to wine appdb and limited. Bit it can be done". The
> >  amount of text to enter for an app entry must be as minimal as
> >  possible: app name, platform (real hardware, emulator), bug number in
> >  bugzilla for possible issues (max. 2 bugs for one entry).
> >  Also, no division for "installs, but doesn't start" or "works only
> >  via manual installation". Either it works, or it doesn't, no
> >  intermediate stages.
> >  As we thought later, it may even be just three columns: appname, ROS
> >  revision, Comments.
> >       
>  I agree with this. All software has bugs. I think the (Works/Does Not
>  Work) is a point of view, of 'can I use this software even with its
>  bugs?'. If yes, then it works. If not, then it does not work.
>
>  Now back to the value thing. Windows costs money, ReactOS can now run
>  hardware GL and Wine DirectX in select virtual machines. If you want
>  to show the value of ReactOS its time to make ReactOS run as many
>  Windows games as possible in these virtual machines on Apple Hardware.
>  This will save OS X users the need to dual boot or acquire a Windows
>  license. Also hardware support is not a big problem, which again helps
>  us. This is the future.
>
>

[Aleksey Bragin]

It's not that bad now, actually that's why I say about starting all  
this compatibility/whatever thing. We know almost all "strange"  
crashes, all of them are bugzilled. Idle time is great too - e.g.  
tower usually runs ReactOS for at least a day in a virtual machine.  
Bad things happen when it comes to certain areas. Like networking in  
your example. Or filesystem (but now it's greatly improved).

I fully agree - stability is a must, but to have a plan, we need to  
see what issues we do have now. And to get them - get test software  
which runs good and could simulate us a real usage case.



On Apr 8, 2009, at 11:24 PM, Timo Kreuzer wrote:

>   One of the current issues that makes ReactOS
> completely useless for  
>  real stuff is not that it cannot run app X or game Y or doesn't  
>  support my network adapter. The most annoying thing is it's  
>  instability.
>  You have no chance to use any of the great features, because most  
>  likely reactos crashes before you are at that point.
>  Try downloading something that is > 300MB, or installing a bigger  
>  application... good luck :(
>  It just crashes way too often. We all know our crappy Cc and yes, I  
>  know win32k also crashes from time to time.
>  And after a crash it's quite likely that things don't work anymore   
>  (for example downloader) and you need to reinstall reactos. If we  
>  could fix that, the system would look much better than it does today.
>  Before a 0.4, we should start a stability offensive.
>  It would also improve developing, because the crashes can really be  
>  a pain when it comes to testing something. Excessive testing is  
>  more or less impossible, because of all the crashes.
>
>  Just my 2 cents,
>  Timo
>

[Ged]

I ran ReactOS in VMWare all day at work yesterday (9 hours), and I used it
for all my web related stuff.
I didn't encounter a single problem all day. I was pleasantly surprised.

I'm going to do this more often now. It's slowly getting into the realms of
being usable for simple every day tasks.



-----Original Message-----
From: ros-dev-bounces(a)reactos.org [mailto:[e-mail]] On
Behalf Of Aleksey Bragin
Sent: 08 April 2009 22:56
To: ReactOS Development List
Subject: Re: [ros-dev] Brainstorming about testing team, roadmap and ReactOS
usability

It's not that bad now, actually that's why I say about starting all  
this compatibility/whatever thing. We know almost all "strange"  
crashes, all of them are bugzilled. Idle time is great too - e.g.  
tower usually runs ReactOS for at least a day in a virtual machine.  
Bad things happen when it comes to certain areas. Like networking in  
your example. Or filesystem (but now it's greatly improved).

I fully agree - stability is a must, but to have a plan, we need to  
see what issues we do have now. And to get them - get test software  
which runs good and could simulate us a real usage case.



On Apr 8, 2009, at 11:24 PM, Timo Kreuzer wrote:

>   One of the current issues that makes ReactOS
> completely useless for  
>  real stuff is not that it cannot run app X or game Y or doesn't  
>  support my network adapter. The most annoying thing is it's  
>  instability.
>  You have no chance to use any of the great features, because most  
>  likely reactos crashes before you are at that point.
>  Try downloading something that is > 300MB, or installing a bigger  
>  application... good luck :(
>  It just crashes way too often. We all know our crappy Cc and yes, I  
>  know win32k also crashes from time to time.
>  And after a crash it's quite likely that things don't work anymore   
>  (for example downloader) and you need to reinstall reactos. If we  
>  could fix that, the system would look much better than it does today.
>  Before a 0.4, we should start a stability offensive.
>  It would also improve developing, because the crashes can really be  
>  a pain when it comes to testing something. Excessive testing is  
>  more or less impossible, because of all the crashes.
>
>  Just my 2 cents,
>  Timo
>  
_______________________________________________
Ros-dev mailing list
Ros-dev(a)reactos.org
http://www.reactos.org/mailman/listinfo/ros-dev

[Cameron Gutman]

To clarify what Timo said (I talked with him earlier today) it is not  
crashing in the network stack. It is crashing in fastfat and Cc.

On Apr 8, 2009, at 5:56 PM, Aleksey Bragin <aleksey(a)reactos.org> wrote:

>   It's not that bad now, actually that's why I
> say about starting all
>  this compatibility/whatever thing. We know almost all "strange"
>  crashes, all of them are bugzilled. Idle time is great too - e.g.
>  tower usually runs ReactOS for at least a day in a virtual machine.
>  Bad things happen when it comes to certain areas. Like networking in
>  your example. Or filesystem (but now it's greatly improved).
>
>  I fully agree - stability is a must, but to have a plan, we need to
>  see what issues we do have now. And to get them - get test software
>  which runs good and could simulate us a real usage case.
>
>
>
>  On Apr 8, 2009, at 11:24 PM, Timo Kreuzer wrote:
>
> >   One of the current issues that makes ReactOS
> > completely useless for
> >  real stuff is not that it cannot run app X or game Y or doesn't
> >  support my network adapter. The most annoying thing is it's
> >  instability.
> >  You have no chance to use any of the great features, because most
> >  likely reactos crashes before you are at that point.
> >  Try downloading something that is > 300MB, or installing a bigger
> >  application... good luck :(
> >  It just crashes way too often. We all know our crappy Cc and yes, I
> >  know win32k also crashes from time to time.
> >  And after a crash it's quite likely that things don't work anymore
> >  (for example downloader) and you need to reinstall reactos. If we
> >  could fix that, the system would look much better than it does today.
> >  Before a 0.4, we should start a stability offensive.
> >  It would also improve developing, because the crashes can really be
> >  a pain when it comes to testing something. Excessive testing is
> >  more or less impossible, because of all the crashes.
> >
> >  Just my 2 cents,
> >  Timo
> >  
>  _______________________________________________
>  Ros-dev mailing list
>  Ros-dev(a)reactos.org
>  http://www.reactos.org/mailman/listinfo/ros-dev

[Aleksey Bragin]

And this is where solution is known and is being developed albeit  
slowly.

On Apr 9, 2009, at 7:36 AM, Cameron Gutman wrote:

>   To clarify what Timo said (I talked with him earlier
> today) it is not
>  crashing in the network stack. It is crashing in fastfat and Cc.
>
>  On Apr 8, 2009, at 5:56 PM, Aleksey Bragin <aleksey(a)reactos.org>  
>  wrote:
>
> > It's not that bad now, actually that's why I say about starting all
> > this compatibility/whatever thing. We know almost all "strange"
> > crashes, all of them are bugzilled. Idle time is great too - e.g.
> > tower usually runs ReactOS for at least a day in a virtual machine.
> > Bad things happen when it comes to certain areas. Like networking in
> > your example. Or filesystem (but now it's greatly improved).
> >
> > I fully agree - stability is a must, but to have a plan, we need to
> > see what issues we do have now. And to get them - get test software
> > which runs good and could simulate us a real usage case.
> >
> >
> >
> > On Apr 8, 2009, at 11:24 PM, Timo Kreuzer wrote:
> >
> >> One of the current issues that makes ReactOS completely useless for
> >> real stuff is not that it cannot run app X or game Y or doesn't
> >> support my network adapter. The most annoying thing is it's
> >> instability.
> >> You have no chance to use any of the great features, because most
> >> likely reactos crashes before you are at that point.
> >> Try downloading something that is > 300MB, or installing a bigger
> >> application... good luck :(
> >> It just crashes way too often. We all know our crappy Cc and yes, I
> >> know win32k also crashes from time to time.
> >> And after a crash it's quite likely that things don't work anymore
> >> (for example downloader) and you need to reinstall reactos. If we
> >> could fix that, the system would look much better than it does  
> >> today.
> >> Before a 0.4, we should start a stability offensive.
> >> It would also improve developing, because the crashes can really be
> >> a pain when it comes to testing something. Excessive testing is
> >> more or less impossible, because of all the crashes.
> >>
> >> Just my 2 cents,
> >> Timo
> >>

[Ged]

If netbooks continue to make an impact, and the ARM processor starts to move
into this territory, I think this is one area we could flourish

Ged.

-----Original Message-----
From: ros-dev-bounces(a)reactos.org [mailto:[e-mail]] On
Behalf Of Steven Edwards
Sent: 08 April 2009 18:26

If you want
to show the value of ReactOS its time to make ReactOS run as many
Windows games as possible in these virtual machines on Apple Hardware.
This will save OS X users the need to dual boot or acquire a Windows
license. Also hardware support is not a big problem, which again helps
us. This is the future.

[Steven Edwards]

On Wed, Apr 8, 2009 at 3:32 PM, Ged <gedmurphy(a)gmail.com> wrote:
>   If netbooks continue to make an impact, and the ARM
> processor starts to move
>  into this territory, I think this is one area we could flourish 
I agree with this too. Given our smaller footprint than Windows I
could see this being a strong point for us.

-- 
Steven Edwards

"There is one thing stronger than all the armies in the world, and
that is an idea whose time has come." - Victor Hugo