SiteMaker Server Status - Service status

SiteMaker Update 10:30 GMT 26 Jan 2012

Josh — Wed, 25 Jan 2012 13:54:00 +0000

We will be updating the core software to version R5.6 at 10:30 GMT, 26 January 2012. There may be a service disruption lasting up to 30 minutes. We recommend users save any changes before this time. After this time, we advise customers to hit reload on their browsers to ensure they have the latest version.

RESOLVED: Service Outage 00:00 GMT 9 Jan 2012

Josh — Mon, 09 Jan 2012 00:17:00 +0000

We are experiencing problems with our firewall routers. Someone is on site at the data center working on this issue. Sorry for the inconvenience.

UPDATE 01:00 GMT: the firewalls are now up and seem to be failing over correctly.

RESOLVED: Service Outage 17:20 GMT 6 Jan 2012

Eirik Pettersen — Fri, 06 Jan 2012 17:32:00 +0000

DETAIL: We are experiencing more problems with our firewall routers. The cause is being investigated. Sorry for the inconvenience

UPDATE 19:20 GMT: Service was restored at 18:07 GMT and the firewall hardware has since been replaced. We expect this to resolve the issue once and for all.

RESOLVED: Service Outage 20:19 GMT 4 Jan 2012

Eirik Pettersen — Wed, 04 Jan 2012 20:25:00 +0000

DETAIL: We are experiencing a problem with our firewall routers. The cause is being investigated.

UPDATE 20:38: It appears our primary firewall layer froze and the secondary firewall failed to pick up the problem, therefore knocking out access to the platform. A forced reboot of the primary caused the secondary to correctly take control and restore access.

RESOLVED: Service Outage 18:30 GMT 30 Dec 2011

Russell — Fri, 30 Dec 2011 19:05:00 +0000

DETAIL: We are experiencing problems with our firewall routers which are causing intermittent brief outages. The cause is being investigated.

UPDATE 20:50: Services have been more stable over the past hour as failover systems have taken control, although brief outages may still be occurring. An engineer is travelling to the data centre to stabilise services further.

UPDATE 23:40: Services have been fully stable for 1 hour now. Enhanced diagnostics have been put into place and monitoring will continue.

COMPLETED: SiteMaker Update 10:30 GMT 15 Dec 2011

Walt — Wed, 14 Dec 2011 18:14:00 +0000

We will be updating the core software to version R5.5.2 at 10:30 GMT, 15 December 2011. There may be a service disruption lasting up to 30 minutes. We recommend users save any changes before this time. After this time, we advise customers to hit reload on their browsers to ensure they have the latest version.

RESOLVED: Service Outage 06:14 GMT 08 Dec 2011

Eirik Pettersen — Thu, 08 Dec 2011 08:26:00 +0000

DETAIL: We have had a rogue process which has filled a temporary database and caused a lockup in the database layer.

UPDATE 06:35: We have killed the process and the databases and platform are recovering.

RESOLVED: Service Outage 18:18 GMT 05 Dec 2011

Eirik Pettersen — Mon, 05 Dec 2011 18:16:00 +0000

DETAIL: It appears the data centre encountered an issue with a power feed to some of our servers.

UPDATE 18:34: Service restored. We will be requesting additional information on the incident from our data centre provider as soon as possible.

RESOLVED: Service Outage 00:30 GMT 19 Nov 2011

Walt — Sat, 19 Nov 2011 16:31:00 +0000

DETAIL: At approximately 00:30 this morning, our Data Centre encountered an issue with a power feed to some of our servers. Normally with the redundant power supply it would fail over instantly and seamlessly and have no impact on supply of services i.e. customers would not have noticed. In this instance the transition hesitated having an impact on our servers.

RESPONSE 01:00: We had to manually restart the servers focusing on those supplying customer websites - our main priority. This took approximately 30 min to fully recover. We then focused on Moonfruit.com which took a further 15 to 20 minutes. Caching issues may have delayed some customers seeing the recovery immediately.

UPDATE 2:45: A full investigation of the incident will be conducted with our Data Centre Management as soon as possible.

RESOLVED: Service Degredation 18:01 GMT 16 Nov 2011

Walt — Wed, 16 Nov 2011 19:33:00 +0000

A number of customers may have intermittent problems accessing moonfruit.com and some accessing their websites.

Update 19:21: We had intermittent issues with our Content Delivery Network provider, Edgecast. That all now seems resolved and all services back to normal.

RESOLVED: Service Degradation 11:00 GMT 9 Nov 2011

Josh — Wed, 09 Nov 2011 12:28:00 +0000

A number of customers may have intermittent problems in accessing their sites.

Cause: There appears to be a DNS issue with a effecting of CDN providers globally, we're currently looking into it.

Update 12:39: the issue seems resolved for most providers. We'll continue to monitor the situation.

COMPLETED: SiteMaker Update 10:30 BST (GMT+1) 13 Oct 2011

Josh — Wed, 12 Oct 2011 16:22:00 +0100

We will be updating the core software to version R5.5 at 10:30 BST (GMT+1), 13th October 2011. There may be a momentary service disruption, but this should be unnoticeable. We recommend users save any changes before this time. After this time, we advise customers to hit reload on their browsers to ensure they have the latest version.

COMPLETED: SiteMaker Update 11:00 BST (GMT+1) 27 Sept 2011

Josh — Mon, 26 Sep 2011 11:20:00 +0100

We will be updating the core software to version R5.4.3 at 11:00 BST (GMT+1), 27 September 2011. There may be a momentary service disruption, but this should be unnoticeable. We recommend users save any changes before this time. After this time, we advise customers to hit reload on their browsers to ensure they have the latest version.

COMPLETED: Scheduled Downtime 11:00-11:30 BST (GMT+1) 13 Sep 2011

Eirik Pettersen — Fri, 09 Sep 2011 18:06:00 +0100

REASON: Database Updates

PLANNED DURATION: 10 minutes

Siteleaders will not be able to login to their sites and visitors to sites will see a unbranded page letting them know that essential maintenance is underway and that they should return later.

UPDATE 12:01: The updates completed successfully at 11:38. There are some maintenance activities following the update that will cause some degradation of performance for the next 30 minutes or so. We apologise for the inconvenience,

UPDATE 12:36: Maintenance is complete, service should be back to normal.

RESOLVED: Service Outage 12:45 BST (GMT+1) 5 Sept 2011

Walt — Mon, 05 Sep 2011 13:09:00 +0100

DETAIL: At 12:45, we started to see a service degradation caused by a database lock-up.

RESPONSE 13:05: We've restored the full service and are commencing an investigation as to what happened. More details once we have established the facts. Apologies for the inconvenience.

DETAIL 14:05: We are encountering further issues which are causing intermittent disruption to the supply of our service. This is being dealt with. Updates will be provided as needed.

RESPONSE 14:32: We located the source of the problem and blocked any further impact. Investigations continue as to the root cause and steps required to minimise further occurrences.

RESOLVED: Service Degradation 10:37 BST (GMT+1) 31 Aug 2011

Eirik Pettersen — Wed, 31 Aug 2011 10:51:00 +0100

DETAIL: We are seeing long database response times, which are slowing down page load speeds. We are investigating the cause.

UPDATE 10:57: A methodical restart of backend services has cleared out the problem.

RESOLVED: Service Degradation 12:00 BST (GMT+1) 26 Aug 2011

Eirik Pettersen — Fri, 26 Aug 2011 12:19:00 +0100

DETAIL: We started seeing unusual activity at 12:00 BST (GMT+1), we are investigating the cause.

RESPONSE 12:45: Service has returned to normal. We experienced a short spike of high load on one of our services which had a cascading effect, causing a backing up of requests. We have made configuration changes to try to mitigate this happening again in future.

RESOLVED: Service Degradation 20:40 BST (GMT+1) 22 Aug 2011

Eirik Pettersen — Mon, 22 Aug 2011 22:04:00 +0100

DETAIL: At 20:40 BST (GMT+1), we are experiencing a spike in traffic resulting in a high load.

RESPONSE 20:52: We've blocked the source of the traffic. The nature of this traffic meant it was not picked up and blocked automatically by our monitoring systems, and required manual intervention. Service should be back to normal.

COMPLETED: SiteMaker Update 11:00 BST (GMT+1) 17 Aug 2011

Josh — Tue, 16 Aug 2011 14:40:00 +0100

We will be updating the core software to version R5.4.1 at 11:00 BST (GMT+1), 17 August 2011. There may be a momentary service disruption, but this should be unnoticeable. We recommend users save any changes before this time. After this time, we advise customers to hit reload on their browsers to ensure they have the latest version.

COMPLETED: SiteMaker Update 10:30 BST (GMT+1) 21 Jul 2011

Josh — Wed, 20 Jul 2011 11:17:00 +0100

We will be updating the core software to version R5.4.0 at 10:30 BST (GMT+1), 21 July 2011. There may be a momentary service disruption, but this should be unnoticeable. We recommend users save any changes before this time. After this time, we advise customers to hit reload on their browsers to ensure they have the latest version.