Re: [ros-dev] [ros-diffs] [akhaldi] 60074: [WINE/TEST] * Introduce a way to properly mark the todos instead of counting them as failures. Brought to you by Thomas Faber.

[Timo Kreuzer]

Hi,

I disagree here. The major purpose of these tests is to let us detect
regressions. But when there is all red anyway, noone notices anything.
It's hard to spot what went wrong, when there is 2275 failures instead
of 2272. Also the results are not lost, they are just moved out of the
way of failing tests. You can still see all the todos.
Being able to see where ros fails, but wine succeeds is a valuable
feature, but it's pretty pointless, when it comes at the cost of being
able to quickly and properly detect regressions. Maybe we can have both
and provide a way to disable this, if someone wants to compare against
wine. Patchbot is our friend.

My wish would be that we only had all green in testbot and whenever
something gets red, Alekseys red telephone starts ringing and he
mobilizes the red army... ;-)

Just my 50 cents,
Timo

Am 14.11.2013 19:18, schrieb James Tabor:
>   Hi.
>
>  This should be removed, we are not wine and I personally work "hard
>  on" getting wine todos fixed in ReactOS! This also breaks the DCE test.
>
>
>  Background, GvG and then the old guard back in the day thought the
>  same way as I do about this issue. We should count the todo's as
>  failures and not bypass these failues. Why? It may point to a critical
>  bug that could break functionality somewhere else. That was the
>  reasoning then and I agree with it. Cheating the tests is just plain
>  cheating......
>
>
>  James
>
>
>
>  _______________________________________________
>  Ros-dev mailing list
>  Ros-dev(a)reactos.org
>  http://www.reactos.org/mailman/listinfo/ros-dev