Hello everybody,
As you might have noticed, I recently did some changes to our Website stuff and Regression Testing System. Finally, it's time to show some results now:
== BuildBot upgrade == Christoph reinstalled the latest version of BuildBot on our Swedish server today, since the previous server system was way too old for such an upgrade. This also had the uncomfortable side-effect of the BuildBot URL changing from http://reactos.org:8010 to http://build.reactos.org:8010, which most of you already seem to have noticed ;-)
Sorry for not giving a note about this before. I thought I could just finish my stuff and then write this E-Mail explaining all changes, but as always, it took longer than expected :-/
== Crash Recovery in rosautotest and sysreg2 == In the recent days, I rewrote rosautotest in C++ for getting a more maintainable code. This update also added a Crash Recovery feature. Together with my updated version of sysreg2, we can now resume the testing process with the next test in case a specific test made ReactOS crash.
== Web Test Interface == As we recently set up a new VM, which will eventually replace our current outdated www.reactos.org server, I finally had a playground for my third component of our testing system: The "testman" Web Interface. It is available at http://79.99.5.181/testman/
Features include finding specific test results through a getbuilds-like interface and comparing them. The Comparison feature can compare up to 5 results side-by-side and show the differences. Clicking on "Show only changed results" will only show the lines that have differences. You can also use Drag & Drop to move the column headers with all associated results to get the table to look like you want it. Clicking a test result will show details, including a log of this specific test.
Just play around with it a bit and you should easily get to know about all features.
Regarding Lone_Rifle's interface at http://reactos.nxserve.net: From what he told me, this one was only meant as a decentral alternative as long as we don't have "testman" on an official server. I'm saying this, so nobody gets the impression that I just wanted to extinguish his work with my implementation. If some people prefer Lone_Rifle's version over mine, it would just need to be adjusted to the new BuildBot URL and those people could still use it.
Comments and suggestions are welcome of course. Not everything works perfect yet (e.g. the Debug Buildslave needs quite some time now as all tests are run), but it should be ready for you to try now.
Best regards,
Colin
Hi
The web test interface is great! <3 <3 <3
Some suggestions: 1) The "compare now" button looks a bit misplaced. It should rather be directly above or below the selections and the "compare last 2 results" should be directly right to it. The complete search block might be put there where the "compare now" button is currently. 2) Maybe regressions (+failed) could be made red rather than green, don't know how this would fit with the red background though ;) ... well just an idea to distinguish good from bad results. 3) I'd like the webtest manager to be above the results in one page - or the results below the interface - like in Lone_Rifles version. (maybe as an option) That way I only need one tab in the browser and don't need 2 tabs or even windows and opening a new for every new compare. I guess it should be no problem to use the full frame below the rectos left side menu.
Regards, Timo
PS: next time just give a hint about the changes on the ml first, and everybody is happy ;)
Colin Finck schrieb:
Hello everybody,
As you might have noticed, I recently did some changes to our Website stuff and Regression Testing System. Finally, it's time to show some results now:
== BuildBot upgrade == Christoph reinstalled the latest version of BuildBot on our Swedish server today, since the previous server system was way too old for such an upgrade. This also had the uncomfortable side-effect of the BuildBot URL changing from http://reactos.org:8010 to http://build.reactos.org:8010, which most of you already seem to have noticed ;-)
Sorry for not giving a note about this before. I thought I could just finish my stuff and then write this E-Mail explaining all changes, but as always, it took longer than expected :-/
== Crash Recovery in rosautotest and sysreg2 == In the recent days, I rewrote rosautotest in C++ for getting a more maintainable code. This update also added a Crash Recovery feature. Together with my updated version of sysreg2, we can now resume the testing process with the next test in case a specific test made ReactOS crash.
== Web Test Interface == As we recently set up a new VM, which will eventually replace our current outdated www.reactos.org server, I finally had a playground for my third component of our testing system: The "testman" Web Interface. It is available at http://79.99.5.181/testman/
Features include finding specific test results through a getbuilds-like interface and comparing them. The Comparison feature can compare up to 5 results side-by-side and show the differences. Clicking on "Show only changed results" will only show the lines that have differences. You can also use Drag & Drop to move the column headers with all associated results to get the table to look like you want it. Clicking a test result will show details, including a log of this specific test.
Just play around with it a bit and you should easily get to know about all features.
Regarding Lone_Rifle's interface at http://reactos.nxserve.net: From what he told me, this one was only meant as a decentral alternative as long as we don't have "testman" on an official server. I'm saying this, so nobody gets the impression that I just wanted to extinguish his work with my implementation. If some people prefer Lone_Rifle's version over mine, it would just need to be adjusted to the new BuildBot URL and those people could still use it.
Comments and suggestions are welcome of course. Not everything works perfect yet (e.g. the Debug Buildslave needs quite some time now as all tests are run), but it should be ready for you to try now.
Best regards,
Colin
Ros-dev mailing list Ros-dev@reactos.org http://www.reactos.org/mailman/listinfo/ros-dev
Timo Kreuzer wrote:
The web test interface is great! <3 <3 <3
Thanks! :-)
Some suggestions:
- The "compare now" button looks a bit misplaced. It should rather be
directly above or below the selections and the "compare last 2 results" should be directly right to it. The complete search block might be put there where the "compare now" button is currently.
Well, the idea behind this is: Do all the selections on the left and then click "Compare now" on the right to do the comparison. Therefore the search box is also on the left, since you can do selections there as well.
I don't have an idea how you want the search box including all the results on the right.... that would exceed the resolution of the widest wide screen ;-)
- Maybe regressions (+failed) could be made red rather than green,
don't know how this would fit with the red background though ;) ... well just an idea to distinguish good from bad results.
Well, currently all test results get the same color and I have no idea how I came to using green for this ;-) And yes, red on red usually doesn't look that well, any other idea? :-P
- I'd like the webtest manager to be above the results in
one page - or the results below the interface - like in Lone_Rifles version. (maybe as an option) That way I only need one tab in the browser and don't need 2 tabs or even windows and opening a new for every new compare. I guess it should be no problem to use the full frame below the rectos left side menu.
Ok, that can be done. Added it to the ToDo list.
PS: next time just give a hint about the changes on the ml first, and everybody is happy ;)
I'll do for sure :-)
Best regards,
Colin
Colin Finck schrieb:
Some suggestions:
- The "compare now" button looks a bit misplaced. It should rather be
directly above or below the selections and the "compare last 2 results" should be directly right to it. The complete search block might be put there where the "compare now" button is currently.
Well, the idea behind this is: Do all the selections on the left and then click "Compare now" on the right to do the comparison. Therefore the search box is also on the left, since you can do selections there as well.
I don't have an idea how you want the search box including all the results on the right.... that would exceed the resolution of the widest wide screen ;-)
Ok I got it now. Didn't try the search so far... Now with the search you have 2 lists of revisions. IMO one list should be sufficient. Just put the search form there, initialize it with the revision range of the last 5 revisions and dispay these below. I added a screenie, I think it optimizes the interface a little. Just an idea....
Incidentally Colin, did those changes muck up the forum admin panel? Cause I can't seem to get to it.
No, this is a result of other changes done last evening :-P
Aleksey, please update the following files with the latest versions from trunk:
forum/common.php forum/includes/auth/auth_roscms.php forum/styles/roscms/template/overall_header.html
After that, the Admin Panel should be ok again. If you don't have time for that now, I'll probably be online this evening to bug you about it ;-)
Best regards,
Colin
_____
From: ros-dev-bounces@reactos.org [mailto:ros-dev-bounces@reactos.org] On Behalf Of Zachary Gorden Sent: Thursday, March 26, 2009 1:19 AM To: ReactOS Development List Subject: Re: [ros-dev] Recent changes
Incidentally Colin, did those changes muck up the forum admin panel? Cause I can't seem to get to it.
Done, please check.
On Mar 26, 2009, at 7:00 PM, Colin Finck wrote:
No, this is a result of other changes done last evening :-P
Aleksey, please update the following files with the latest versions from trunk:
forum/common.php forum/includes/auth/auth_roscms.php forum/styles/roscms/template/overall_header.html
After that, the Admin Panel should be ok again. If you don't have time for that now, I'll probably be online this evening to bug you about it ;-)
Best regards,
Colin
From: ros-dev-bounces@reactos.org [mailto:ros-dev- bounces@reactos.org] On Behalf Of Zachary Gorden Sent: Thursday, March 26, 2009 1:19 AM To: ReactOS Development List Subject: Re: [ros-dev] Recent changes
Incidentally Colin, did those changes muck up the forum admin panel? Cause I can't seem to get to it.
Please use "http://build.reactos.org" as new URL - without a port number.
Regards, Christoph von Wittich
Christoph von Wittich wrote:
Please use "http://build.reactos.org" as new URL - without a port number.
But that page auto-redirects to the URL with the port number. Additionally, links like http://build.reactos.org:8010/builders/ReactOS_(Debug)/builds/13/steps/test/ logs/stdio/text also work with the port number only.
Best regards,
Colin
-
Aleksey Bragin -
Christoph von Wittich -
Colin Finck -
Timo Kreuzer -
Zachary Gorden