Some upcoming changes

[Colin Finck]

Hello all,

I just wanted to announce some upcoming changes. Comments are appreciated.


Official removal of i486 support
---------------------------------
The Mingw-w64 project held an internal meeting today. One of the decisions
was to remove support for architectures older than i586 from their code.
As we already use some of their runtime code and loosely planned to switch
to a mingw-w64 x86/x64 multilib toolchain once that is released and stable
(which should be the case in gcc 4.5.x), we will be affected by this change,
too.

Of course, the Mingw-w64 code would just add another problem to the pot of
i486 incompatibilities.
For example, some of our assembly code makes use of the CMPXCHG8B
instruction (introduced in the Pentium).

Though I don't know anybody, who tried a recent ReactOS build on an i486
machine, I also never found a statement about the official removal of i486
support.
So as long as nobody objects, let's officially declare i486 support
abandoned now.


ISO cleaning on iso.reactos.org
--------------------------------
As we were right before running out of space on the ISO storage server (in
fact, that would have happened when Aleksey was on his planned holiday trip
:D), I've installed a script to clean old ISOs automatically now.
The script will remove bootcd-dbg ISOs older than 2 years and all other ISOs
older than half a year. This should free enough space while keeping enough
important ISOs for tracking down old regressions.


Testman log cleaning
---------------------
Another service that takes more space than you might think is Testman. Logs
for ~1300 revisions currently take around 2 GB.
My plan would be to delete logs (not the actual test result numbers) older
than half a year to keep the used space for them on an almost constant
level. If nobody objects, such a script will be installed in the upcoming
days.


Best regards,

Colin

[Alex Ionescu]

Can we please 7zip logs instead of deleting them? Should give you
massive improvements (especially since A) it's text B) it contains
highly-repeatable file/function names).

Thanks!

Best regards,
Alex Ionescu



On Sat, Aug 15, 2009 at 7:17 PM, Colin Finck<mail(a)colinfinck.de> wrote:
>   Hello all,
>
>  I just wanted to announce some upcoming changes. Comments are appreciated.
>
>
>  Official removal of i486 support
>  ---------------------------------
>  The Mingw-w64 project held an internal meeting today. One of the decisions
>  was to remove support for architectures older than i586 from their code.
>  As we already use some of their runtime code and loosely planned to switch
>  to a mingw-w64 x86/x64 multilib toolchain once that is released and stable
>  (which should be the case in gcc 4.5.x), we will be affected by this change,
>  too.
>
>  Of course, the Mingw-w64 code would just add another problem to the pot of
>  i486 incompatibilities.
>  For example, some of our assembly code makes use of the CMPXCHG8B
>  instruction (introduced in the Pentium).
>
>  Though I don't know anybody, who tried a recent ReactOS build on an i486
>  machine, I also never found a statement about the official removal of i486
>  support.
>  So as long as nobody objects, let's officially declare i486 support
>  abandoned now.
>
>
>  ISO cleaning on iso.reactos.org
>  --------------------------------
>  As we were right before running out of space on the ISO storage server (in
>  fact, that would have happened when Aleksey was on his planned holiday trip
>  :D), I've installed a script to clean old ISOs automatically now.
>  The script will remove bootcd-dbg ISOs older than 2 years and all other ISOs
>  older than half a year. This should free enough space while keeping enough
>  important ISOs for tracking down old regressions.
>
>
>  Testman log cleaning
>  ---------------------
>  Another service that takes more space than you might think is Testman. Logs
>  for ~1300 revisions currently take around 2 GB.
>  My plan would be to delete logs (not the actual test result numbers) older
>  than half a year to keep the used space for them on an almost constant
>  level. If nobody objects, such a script will be installed in the upcoming
>  days.
>
>
>  Best regards,
>
>  Colin
>
>
>  _______________________________________________
>  Ros-dev mailing list
>  Ros-dev(a)reactos.org
>  http://www.reactos.org/mailman/listinfo/ros-dev
>

[Colin Finck]

Alex Ionescu wrote:
>   Can we please 7zip logs instead of deleting them?
As the logs are not simple text files, but stored in a database for showing
them together with other test information (see e.g.
http://backend.reactos.org/testman/detail.php?id=341420), I've used MySQL's
COMPRESS/UNCOMPRESS instead of 7-Zip now.
In my opinion, the higher memory requirements and system load of 7-Zip also
don't outweigh the compression ratio for text files, which shouldn't be that
much better compared to a zlib-based algorithm (as it's used by
COMPRESS/UNCOMPRESS). Keep in mind that everybody shall still be able to
easily access these logs over the web interface, so they have to be
uncompressed on-the-fly.

But all in all, using compression here was a good idea, we're down to 190 MB
now. :-)

Though I'm still not in favor of keeping logs of ancient revisions forever.
It's okay if we extend the limit to 2-3 years, but does anybody really need
even older logs?

Best regards,

Colin

[Alex Ionescu]

How about archiving the older logs in a format that may be a one-time
CPU expense, but never have to worry about uncompressing? Ie: I don't
want really old logs accessible through the UI, but just available by
developer request.

Best regards,
Alex Ionescu



On Mon, Aug 17, 2009 at 2:36 AM, Colin Finck<mail(a)colinfinck.de> wrote:
>   Alex Ionescu wrote:
> >   Can we please 7zip logs instead of deleting them?
>  As the logs are not simple text files, but stored in a database for showing
>  them together with other test information (see e.g.
>  http://backend.reactos.org/testman/detail.php?id=341420), I've used MySQL's
>  COMPRESS/UNCOMPRESS instead of 7-Zip now.
>  In my opinion, the higher memory requirements and system load of 7-Zip also
>  don't outweigh the compression ratio for text files, which shouldn't be that
>  much better compared to a zlib-based algorithm (as it's used by
>  COMPRESS/UNCOMPRESS). Keep in mind that everybody shall still be able to
>  easily access these logs over the web interface, so they have to be
>  uncompressed on-the-fly.
>
>  But all in all, using compression here was a good idea, we're down to 190 MB
>  now. :-)
>
>  Though I'm still not in favor of keeping logs of ancient revisions forever.
>  It's okay if we extend the limit to 2-3 years, but does anybody really need
>  even older logs?
>
>  Best regards,
>
>  Colin
>
>
>  _______________________________________________
>  Ros-dev mailing list
>  Ros-dev(a)reactos.org
>  http://www.reactos.org/mailman/listinfo/ros-dev
>

[Steven Edwards]

On Mon, Aug 17, 2009 at 6:09 PM, Alex Ionescu<ionucu(a)videotron.ca> wrote:
>   How about archiving the older logs in a format that
> may be a one-time
>  CPU expense, but never have to worry about uncompressing? Ie: I don't
>  want really old logs accessible through the UI, but just available by
>  developer request. 
I think keeping this data around is a good idea. Besides the technical
reference reasons a given developer might have, I think there is a
good case that can be made via the evolution of conformance in a legal
sense. Let's say vendor comes along and says 'this code is stolen,
reversed, whatever', we've not only got the code in revision history
but we've got a clear example of how 'original' and non-compliant
large parts of the system were and how the evolved. Its kind of like
saying 'if there is tainted code somewhere, then why the hell is the
system not perfect by now'. We would be able to point to another
source of history showing its evolution to perfection.

-- 
Steven Edwards

"There is one thing stronger than all the armies in the world, and
that is an idea whose time has come." - Victor Hugo

[Colin Finck]

Steven Edwards wrote:
>   I think keeping this data around is a good idea.
> Besides the technical
>  reference reasons a given developer might have, I think there is a
>  good case that can be made via the evolution of conformance in a legal
>  sense. 
Good point, I haven't thought about that indeed.
So yes, besides having the log data easily accessible from the web interface
(which will be deleted if older than 1-2 years), I'll implement an unlimited
internal file archive of that log data in the next few days.

Apart from that, I have just migrated the Testman data to
http://reactos.org/testman.
The old links http://79.99.5.181/testman and
http://backend.reactos.org/testman (both lead to the same server) won't work
anymore, but show you the new URL as well.

Best regards,

Colin