Hi all,
This is an acknowledgment and informative email. Since we migrated the ReactOS website to Drupal, you all have spotted critical performances issues. We did our best to address most of them in the following days to ensure smooth browsing on the website. Unfortunately we could not fix them all. Even worse, nowadays, ReactOS website faces a kind of "shutdown" everyday (approximately between 2:45 AM CET and 4:45 AM CET). This is a really long and critical downtime that affects mainly US users due to timezones. Be sure we are aware about it.
Unfortunately, this is not something we can fix from a day to the following. This is due to our backup policies that completely lock the MySQL server to ensure consistent backups. We are currently working on getting this fixed as fast as possible. But this will require (besides the rest) a two hardware upgrades on our infrastructure. Indeed, Drupal stresses our infrastructure much more than old RosCMS. And we have to fully rethink our web infrastructure and redistribute it, which we cannot do at the moment due to hardware constraints. Upgrading our hardware will be in our first steps (obviously, expect downtimes at that moment). Then, we will finally be able to replicate our MySQL server to perform load-balancing and prevent shutdowns on backup.
We, the sysadmins team and the website team, agreed on a plan about how to enhance and strengthen the infrastructure, and thus fix all the issues we are currently facing. This will take some time, but we are working together to get ReactOS website back into a decent shape.
Be assured that we are doing our best to make your place nice. Meanwhile, please forgive us the caused inconvenience.
With my best regards, on behalf of the guys working behind the scene,
Thanks! Your work is very appreciated! Please make sure that you keep an eye on the performance of both doxygen search and git grep, which have suffered severly. These are essential not only for work on reactos, but also for people outside the project. (I use it on a daily basis at work.)
Thanks for your work, Timo
Hi,
I have to ask, with all the money being spent on hardware & etc, doesn't it make more sense to just host on AWS or Rackspace?
-- Best regards, Alex Ionescu
-----Original Message----- From: ros-general-bounces@reactos.org [mailto:ros-general-bounces@reactos.org] On Behalf Of Pierre Schweitzer Sent: Friday, November 15, 2013 12:24 PM To: ReactOS General List; ReactOS Development List Subject: [ros-general] ReactOS website issues
Hi all,
This is an acknowledgment and informative email. Since we migrated the ReactOS website to Drupal, you all have spotted critical performances issues. We did our best to address most of them in the following days to ensure smooth browsing on the website. Unfortunately we could not fix them all. Even worse, nowadays, ReactOS website faces a kind of "shutdown" everyday (approximately between 2:45 AM CET and 4:45 AM CET). This is a really long and critical downtime that affects mainly US users due to timezones. Be sure we are aware about it.
Unfortunately, this is not something we can fix from a day to the following. This is due to our backup policies that completely lock the MySQL server to ensure consistent backups. We are currently working on getting this fixed as fast as possible. But this will require (besides the rest) a two hardware upgrades on our infrastructure. Indeed, Drupal stresses our infrastructure much more than old RosCMS. And we have to fully rethink our web infrastructure and redistribute it, which we cannot do at the moment due to hardware constraints. Upgrading our hardware will be in our first steps (obviously, expect downtimes at that moment). Then, we will finally be able to replicate our MySQL server to perform load-balancing and prevent shutdowns on backup.
We, the sysadmins team and the website team, agreed on a plan about how to enhance and strengthen the infrastructure, and thus fix all the issues we are currently facing. This will take some time, but we are working together to get ReactOS website back into a decent shape.
Be assured that we are doing our best to make your place nice. Meanwhile, please forgive us the caused inconvenience.
With my best regards, on behalf of the guys working behind the scene,
-- Pierre Schweitzer<pierre at reactos.org> System Administrator ReactOS Foundation
Hi Alex,
Alex Ionescu ionucu@videotron.ca wrote:
I have to ask, with all the money being spent on hardware & etc, doesn't it make more sense to just host on AWS or Rackspace?
We're currently tackling an administration/configuration problem rather than the lack of proper infrastructure. Hosting somewhere else would still require choosing the right tools and tuning the configuration. This is what we're currently doing.
Our primary services like website, mail and SVN are already hosted in a German datacenter with modern hardware and very competitive monthly prices. I have no personal experiences with AWS or Rackspace, but I see that they are billing by service and traffic. No option for us as we need fixed monthly costs for hosting. On top of this, if we choose e.g. a web server and a DBMS at AWS, we couldn't use these hosting resources for any other services. Actually, such a hosting plan would make our infrastructure even less flexible.
The money we recently spent on hardware is totally unrelated to this. We already don't have to pay monthly costs for hosting such own servers, and I believe this is unbeatable ;)
- Colin
Hi,
so the backup does need 2 hours? How big is our database?! Thats very, very long... A first idea would be mysqlhotcopy, which does the locking etc. for you - and copy the raw database files (instead of making sql-files which can take ages). Informations are under http://dev.mysql.com/doc/refman/5.1/en/mysqlhotcopy.html . It can do things e.g. skipping the index files which can be rebuilt, flush the logs etc.
Another approach is using a lvm under the database-files. This way you get at least a consistent version which are on the HDDs. Yes, you loose all information which are only in the RAM, but that shouldn't be too much - und if you need to replay backups you loose always by average 12 hours - so 5-10 minutes more ore less aren't that important ;)
Best regards, Michael Fritscher
-
Alex Ionescu -
Colin Finck -
Michael Fritscher -
Pierre Schweitzer -
Timo Kreuzer