There was some confusion and disagreement about the Testing Coordinator function description recently. The best way to solve it seems to be a vote. With the new voting procedure established and 0.2.8 (almost) out the door, this seems a good time to start the voting process. Our (draft) constitution calls for a 5 day discussion period, followed by a 5 day voting period. This email starts the 5 day discussion period. Preferred location for the discussion is the ros-dev mailing list, since all developers have easy access to that. At the end of the discussion, I'll post a call for votes to the mailing list and create the vote on the forum.
We have two proposals for the TC function description, A and B. A is the original function description. Proposal B is almost the same, the only difference is that the TC does not have the authority to block a release.
This is the motion I would like to vote on:
"Do you want to: A) set the function description for the TC role as described in http://www.reactos.org/wiki/index.php/ReactOS_Testing_Coordinator or B) set the function description for the TC role as described in http://www.reactos.org/wiki/index.php/ReactOS_Testing_Coordinator_Alternate"
Gé van Geldorp.
Ge van Geldorp wrote:
There was some confusion and disagreement about the Testing Coordinator function description recently. The best way to solve it seems to be a vote. With the new voting procedure established and 0.2.8 (almost) out the door, this seems a good time to start the voting process. Our (draft) constitution calls for a 5 day discussion period, followed by a 5 day voting period. This email starts the 5 day discussion period. Preferred location for the discussion is the ros-dev mailing list, since all developers have easy access to that. At the end of the discussion, I'll post a call for votes to the mailing list and create the vote on the forum.
We have two proposals for the TC function description, A and B. A is the original function description. Proposal B is almost the same, the only difference is that the TC does not have the authority to block a release.
This is the motion I would like to vote on:
"Do you want to: A) set the function description for the TC role as described in http://www.reactos.org/wiki/index.php/ReactOS_Testing_Coordinator or B) set the function description for the TC role as described in http://www.reactos.org/wiki/index.php/ReactOS_Testing_Coordinator_Alternate"
Gé van Geldorp.
Under "Alternate" URL, Section "Powers", 2nd paragraph, you have: "...hinder any legitimate efforts...". I think the word legitimate is too vague here, and the sentence should be changed to be more precise.
I'm having a difficult time coming up with an alternate suggestion, so I figured I'd think aloud here... All powers granted by voting people into positions are temporary. We are not setting anybody up as god. So in either scenario, there has to be the ability to overrule the TC. Furthermore, the only time the TC's "prioritization" of a bug is during a release process. For those reasons, based on the two proposals as they are, I think the voting should be between:
A) 3 devs have the power to overrule the TC regarding a bug's priority for the purposes of a release.
B) it takes a majority vote to overrule the TC regarding a bug's priority for the purposes of a release.
Moreover, I'm thinking maybe there's another way to skin this cat...
Instead of the TC having to fight to prevent a release he thinks is necessary, we could have a slightly different procedure:
1) RC is ready for FF. Asks TC for revision to branch. If TC feels there are too many blockers for FF, he presents a list of blockers to the list. ( You can't really set up a forum vote for this - forum votes only allow you to pick one item from a list. ). We'd have to decide here exactly what kind of a "vote" this would be - should 3 devs in agreement on a single blocker be able to enforce the TC's block, or what?
2) The CF process would work the same as FF, except without the branching, as the code is already branched.
The advantage of doing it this way is we already have a list of "known bugs" we can apply to the release if we decide to go forward.
Anyways, just some random musings...
Peace
From: Royce Mitchell III
Furthermore, the only time the TC's "prioritization" of a bug is during a release process. For those reasons, based on the two proposals as they are, I think the voting should be between:
A) 3 devs have the power to overrule the TC regarding a bug's priority for the purposes of a release.
B) it takes a majority vote to overrule the TC regarding a bug's priority for the purposes of a release.
The whole point of proposal B is to not give the TC special powers with respect to a release. In proposal A, the TC decides whether to publish a release or not (with a possibility to overrule that decision). Personally, I think that's fundamentally wrong. In my view, the developers are responsible for the quality of the code, not the TC. If a dev introduces a bug which is not found during testing, it's still the devs "fault", not the "fault" of the testing team. With the devs responsible for the quality, they should be the primary decision making group on whether to release or not. Of course, you don't have to agree with my point of view, that's why there are two proposals up for vote.
Instead of the TC having to fight to prevent a release he thinks is necessary, we could have a slightly different procedure:
- RC is ready for FF. Asks TC for revision to branch. If TC
feels there are too many blockers for FF, he presents a list of blockers to the list. ( You can't really set up a forum vote for this - forum votes only allow you to pick one item from a list. ). We'd have to decide here exactly what kind of a "vote" this would be - should 3 devs in agreement on a single blocker be able to enforce the TC's block, or what?
- The CF process would work the same as FF, except without
the branching, as the code is already branched.
(For the benefit of those wondering about the acronyms: FF = Feature Freeze, point where we make the release branch, no new features go in the branch, only bug fixes. CF = Code Freeze, only major bugs are fixed during this stage, every commit needs to be discussed on the mailinglist first). The procedure you suggest has the danger of postponing releases indefinetly. There will always be a feature just around the corner (networking, anyone?) or a serious bug. I believe we should go back to time-based releases. Every two months, the release coordinator publishes a release schedule: "we're going to branch for 0.x.x and enter feature freeze on xx/xx, with code freeze expected on xx/xx and final release on xx/xx". So, it's possible that we end up branching with a serious bug present. That bug will be in the FF (or Preview or whatever you want to call it, the thing we currently call RC1) which is published on SourceForge. The point of the FF is not to have a perfect .iso out there, but to get extra testers to find other bugs. The TC is very important especially during the feature freeze phase. It would be good to have a list of serious problems (hopefully that list is empty, by the way) when we enter feature freeze and be able to cross the problems away as we fix them. It's just that I believe we shouldn't make the decision to branch based on the problem list, it should be time-based instead. Now, entering code freeze is another matter. That's basically the point where we say "ok, what we have in the release branch is good enough for an official release". That's why I said "code freeze expected on xx/xx" above, the date of a code freeze can and should shift if serious problems are still present.
Peace
And hapiness to all.
Gé.
-----Original Message----- From: ros-dev-bounces@reactos.org [mailto:ros-dev-bounces@reactos.org] On Behalf Of Ge van Geldorp Sent: 25. oktober 2005 23:17 To: 'ReactOS Development List' Subject: [ros-dev] TC function description: opening discussion prior to vote
There was some confusion and disagreement about the Testing Coordinator function description recently. The best way to solve it seems to be a vote. With the new voting procedure established and 0.2.8 (almost) out the door, this seems a good time to start the voting process. Our (draft) constitution calls for a 5 day discussion period, followed by a 5 day voting period. This email starts the 5 day discussion period. Preferred location for the discussion is the ros-dev mailing list, since all developers have easy access to that. At the end of the discussion, I'll post a call for votes to the mailing list and create the vote on the forum.
[CSH] 7 days actually, unless specified otherwise.
We have two proposals for the TC function description, A and B. A is the original function description. Proposal B is almost the same, the only difference is that the TC does not have the authority to block a release.
This is the motion I would like to vote on:
"Do you want to: A) set the function description for the TC role as described in http://www.reactos.org/wiki/index.php/ReactOS_Testing_Coordinator or B) set the function description for the TC role as described in http://www.reactos.org/wiki/index.php/ReactOS_Testing_Coordinator_Alternate"
Gé van Geldorp.
[CSH] Both A and B would violate the constitution if we agree to it. It says a vote can override any decision or action made by any project member.
This paragraph violates it: "The TC has final say over the priority of a bug. By his election, the developers and board members have agreed to trust in his judgement related to bug priorities, and must not hinder any legitimate efforts that the TC is attempting in order to prioritize bugs. "
I believe the whole paragraph is void. The TC (and any other project member) can do almost anything (including changing priority of a bug and attempting to block a release). All the draft constitution asks for, is that every registered project member is given an opportunity to object (and call for a vote on the subject if needed).
I'm not sure what this sentence implies: "The TC's tools include complete access over Bugzilla, as well as a team of testers which respond directly to the TC."
Casper
From: Casper Hornstrup
Our (draft) constitution calls for a 5 day discussion period, followed by a 5 day voting period.
[CSH] 7 days actually, unless specified otherwise.
Damn, I should have checked instead of relying on my memory. Thanks for the correction, we now have 6 days to go in the discussion period which will be followed by a 7 day voting period.
[CSH] Both A and B would violate the constitution if we agree to it. It says a vote can override any decision or action made by any project member.
This paragraph violates it: "The TC has final say over the priority of a bug. By his election, the developers and board members have agreed to trust in his judgement related to bug priorities, and must not hinder any legitimate efforts that the TC is attempting in order to prioritize bugs. "
I believe the whole paragraph is void. The TC (and any other project member) can do almost anything (including changing priority of a bug and attempting to block a release). All the draft constitution asks for, is that every registered project member is given an opportunity to object (and call for a vote on the subject if needed).
Royce also pointed out that that paragraph is not very clear. We should probably drop it in proposal B. I don't feel very comfortable making changes to proposal A, I didn't write and don't feel responsible for it. I hope someone else will pick up this issue with respect to A.
Gé van Geldorp.
One day late (I still like coding better than these administrative tasks :-)) I call for a vote on the TC function description. The vote will run for 7 days and ends 8 Nov. I will create the vote on the forum.
I'm slightly amending the proposals to vote on, after Casper pointed out both of the proposals would violate the (draft) constitution. In proposal A (which gives the TC power to block a release) "the approval from a board member" is no longer required to override the TC's decision to block a release. In proposal B (no blocking power) the sentence "The TC has final say over the priority of a bug. By his election, the developers and board members have agreed to trust in his judgement related to bug priorities, and must not hinder any legitimate efforts that the TC is attempting in order to prioritize bugs." was dropped.
In order to preserve the full text of the proposals for our children and grandchildren, I'm including them below so they will get archived.
This is the resolution to vote on:
Do you want to: A) set the function description for the TC role as described below under "Proposal A" or B) set the function description for the TC role as described below under "Proposal B"
Happy voting, Gé van Geldorp.
========= Proposal A =========
Mandate
The Testing Coordinator's (TC) job is to ensure continous quality releases of ReactOS builds and to periodically verify the current status of the SVN "HEAD" Branch in order to quickly identify regressions. The TC's job becomes even more critical as we approach an official release, as every application known to work in a previous release must continue working, and the fixing of older bugs needs to be assured in order to ensure a steady stream of quality increase between releases. The TC's tools include complete access over Bugzilla, as well as a team of testers which respond directly to the TC.
Term
1 year renewable.
Powers
Since one of the TC's primary and most important tools is the Bugzilla Database, the TC is granted complete administrative control over the Database, with the power to create, delete, modify any bugs he chooses, as well as the ability to perform administrative maintenance tools such as adding a new revision to the databases's list, handling developer emails, assigning bugs, etc. Due to the sensitive nature of such control, the TC must be properly trained in the usage of the administrative interface, in order to avoid accidental damage to the DB.
The TC has final say over the priority of a bug. The TC can decide as his will which bugs to promote to Blocker status (thereby blocking a new release), or to demote bugs below that status. Although the TC is recommended to specify a reason for his judgement, his decisions should not be the cause of endless flame wars and challenges. By his election, the developers and board members have agreed to trust in his judgement related to bug priorities, and must not hinder any legitimate efforts that the TC is attempting in order to prioritize bugs. However, in the rare case where it is clear to a majority of developers that the decision to block a release is unwarranted, a group of no less then three developers may override the TC's decision. Such actions should never have to happen however, and communicaton is highly recommended in order to reach a moderated debate.
Obligations
Since the TC's job is critical to the success of ReactOS on the public scene, as well as a smooth development environment internally, unhindered by the constant search of 3000-revision old regressions, there are some obligations which the TC has towards the ReactOS Team.
One of the key obligations of the TC is constant contact with the developer team. The TC must be present in daily development activities as much as possible, to ensure that he is constantly aware of new bugs, difficulties in fixing some bugs, and other design or architectural decisions which might impact the severity and/or presence of a bug or regression. This is to avoid cases where the entire developer team is aware of a voluntary temporary regression, but the TC knows nothign about it and blocks the release.
In order to fulfill his actual mandate, the TC is required to test the current SVN at an acceptable pace. It is hard to give an exact number of how many revisions can pass beteween a new test, as sometimes there can be 40 consecutive revisions that merely fix some naming, while other times, a single revision can change everything. Therefore, it is up to the TC to be in contact with the dev team, and also his personal judgement to determine when a certain treshold of potentially regressive changes have been done.
Finally, the TC should have an active list of applications, tools, and testing environments used to test the builds. He should provide this list publically so the other members of the team can collaborate, comment on, and potentially verify and more important duplicate results. This will be key to allowing developers to easily fix regressions or bugs. The TC should set up a page or alternative method of visual real-time communication showing the current applications tested/in testing and wether any flaws have been noted. Preferably, specific features of each application should be tested. An example is listed below:
CURRENT REV: 25000 - Abiword - Menu System V - File/Open System V - Toolbars, docking, and other graphical elements F - Regression. Docking a toolbar makes it disappear - Regedit - Deleting a key V - Creating a key F - Known bug. Creating a key cannot be done.
========= Proposal B =========
Mandate
The Testing Coordinator's (TC) job is to ensure continous quality releases of ReactOS builds and to periodically verify the current status of the SVN "HEAD" Branch in order to quickly identify regressions. The TC's job becomes even more critical as we approach an official release, as every application known to work in a previous release must continue working, and the fixing of older bugs needs to be assured in order to ensure a steady stream of quality increase between releases. The TC's tools include complete access over Bugzilla, as well as a team of testers which respond directly to the TC.
Term
1 year renewable.
Powers
Since one of the TC's primary and most important tools is the Bugzilla Database, the TC is granted complete administrative control over the Database, with the power to create, delete, modify any bugs he chooses, as well as the ability to perform administrative maintenance tools such as adding a new revision to the databases's list, handling developer emails, assigning bugs, etc. Due to the sensitive nature of such control, the TC must be properly trained in the usage of the administrative interface, in order to avoid accidental damage to the DB.
Obligations
Since the TC's job is critical to the success of ReactOS on the public scene, as well as a smooth development environment internally, unhindered by the constant search of 3000-revision old regressions, there are some obligations which the TC has towards the ReactOS Team.
One of the key obligations of the TC is constant contact with the developer team. The TC must be present in daily development activities as much as possible, to ensure that he is constantly aware of new bugs, difficulties in fixing some bugs, and other design or architectural decisions which might impact the severity and/or presence of a bug or regression. This is to avoid cases where the entire developer team is aware of a voluntary temporary regression, but the TC knows nothign about it and blocks the release.
In order to fulfill his actual mandate, the TC is required to test the current SVN at an acceptable pace. It is hard to give an exact number of how many revisions can pass beteween a new test, as sometimes there can be 40 consecutive revisions that merely fix some naming, while other times, a single revision can change everything. Therefore, it is up to the TC to be in contact with the dev team, and also his personal judgement to determine when a certain treshold of potentially regressive changes have been done.
Finally, the TC should have an active list of applications, tools, and testing environments used to test the builds. He should provide this list publically so the other members of the team can collaborate, comment on, and potentially verify and more important duplicate results. This will be key to allowing developers to easily fix regressions or bugs. The TC should set up a page or alternative method of visual real-time communication showing the current applications tested/in testing and wether any flaws have been noted. Preferably, specific features of each application should be tested. An example is listed below:
CURRENT REV: 25000 - Abiword - Menu System V - File/Open System V - Toolbars, docking, and other graphical elements F - Regression. Docking a toolbar makes it disappear - Regedit - Deleting a key V - Creating a key F - Known bug. Creating a key cannot be done.
Ge van Geldorp wrote:
In proposal B (no blocking power) the sentence "The TC has final say over the priority of a bug. By his election, the developers and board members have agreed to trust in his judgement related to bug priorities, and must not hinder any legitimate efforts that the TC is attempting in order to prioritize bugs." was dropped.
I agree with everything, including removing "The TC has final say over the priority of a bug." which caused all this, but is it really right to remove:
"By his election, the developers have agreed to trust in his judgement related to bug priorities"? It would seem to me like that's a given. It doesn't represent any grant of power, just states the obvious. We're voting for Foo as TC because we trust he'll be a good TC. Addtionally: "must not hinder any legitimate efforts that the TC is attempting in order to prioritize bugs." is also not an adtional grant of power, but simply specifies that if the TC thinks Bug XXX is low priority, a developer (i'm talking about one, not a majority) shouldn't constantly raise it back to BLOCKER just because he feels this is very important to him. I think this is also normal curteous behaviour... once again, I'm not saying that the TC's decision should be absolute, just that if the TC and ONE developer shouldn't have to go through priority wars (unless there is a majority). I hope this makes sense.. I'm waiting for your comments.
Best regards, Alex Ionescu
From: Alex Ionescu
I agree with everything, including removing "The TC has final say over the priority of a bug." which caused all this, but is it really right to remove:
"By his election, the developers have agreed to trust in his judgement related to bug priorities"? It would seem to me like that's a given. It doesn't represent any grant of power, just states the obvious. We're voting for Foo as TC because we trust he'll be a good TC.
If it's a given and obvious, there's no reason to put it in? The problem with the sentence is that it can be read to imply that the TCs decisions in this regard cannot be overridden by a developers vote. Since it didn't seem to add anything but could be misinterpreted, the simplest solution was to just drop it.
Addtionally: "must not hinder any legitimate efforts that the TC is attempting in order to prioritize bugs." is also not an adtional grant of power, but simply specifies that if the TC thinks Bug XXX is low priority, a developer (i'm talking about one, not a majority) shouldn't constantly raise it back to BLOCKER just because he feels this is very important to him. I think this is also normal curteous behaviour...
Royce pointed out (http://www.reactos.org/archives/public/ros-dev/2005-October/005779.html) that "legitimate" isn't very precise. Again, since it doesn't add anything (the TC is given administrative control over the Bugzilla installation in the previous paragraph) I dropped it.
GvG
Im not a member of the team, i know, but it seems that there is starting to become an aulful lot of politics involved with the project. I didn't think that ReactOS was that big already...
On 11/1/05, Ge van Geldorp gvg@reactos.org wrote:
From: Alex Ionescu
I agree with everything, including removing "The TC has final say over the priority of a bug." which caused all this, but is it really right to remove:
"By his election, the developers have agreed to trust in his judgement related to bug priorities"? It would seem to me like that's a given. It doesn't represent any grant of power, just states the obvious. We're voting for Foo as TC because we trust he'll be a good TC.
If it's a given and obvious, there's no reason to put it in? The problem with the sentence is that it can be read to imply that the TCs decisions in this regard cannot be overridden by a developers vote. Since it didn't seem to add anything but could be misinterpreted, the simplest solution was to just drop it.
Addtionally: "must not hinder any legitimate efforts that the TC is attempting in order to prioritize bugs." is also not an adtional grant of power, but simply specifies that if the TC thinks Bug XXX is low priority, a developer (i'm talking about one, not a majority) shouldn't constantly raise it back to BLOCKER just because he feels this is very important to him. I think this is also normal curteous behaviour...
Royce pointed out (http://www.reactos.org/archives/public/ros-dev/2005-October/005779.html) that "legitimate" isn't very precise. Again, since it doesn't add anything (the TC is given administrative control over the Bugzilla installation in the previous paragraph) I dropped it.
GvG
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
We used to beat each other until everyone, but one gave up. It wasn't very constructive.
Casper
-----Original Message----- From: ros-dev-bounces@reactos.org [mailto:ros-dev-bounces@reactos.org] On Behalf Of Brian Sent: 2. november 2005 18:35 To: ReactOS Development List Subject: [ros-dev] Re: TC function description: Call for vote
Im not a member of the team, i know, but it seems that there is starting to become an aulful lot of politics involved with the project. I didn't think that ReactOS was that big already...
The the vote on the TC function description is now closed. Final results:
Proposal A 11 votes Proposal B 5 votes
which means proposal A is accepted. I will update the website accordingly (probably tomorrow). This outcome means we also have to change/update the release process document. It is also time to elect a TC now. I'll start two separate email threads for these.
Gé van Geldorp.
-
Alex Ionescu -
Brian -
Casper Hornstrup -
Ge van Geldorp -
Royce Mitchell III