Unexpected outage

Closed | Jan 20, 2022 | 14:31 GMT+02:00

The issue has been resolved and closed.

An update that went to production during the morning (2022-01-20) included a new feature which under a heavy load caused database performance issues. It cascaded and caused performance issues which appeared to users as 504 Gateway Timeout or as a websocket connection error.

When the issue was identified actions were taken to resolve it as soon as possible.

We apologise for any inconvenience it might have caused and thank you for your patience and understanding.

+Show history

• Resolved | Jan 20, 2022 | 10:46 GMT+02:00

  Howspace is back online. We have applied a fix that resolved the issue.
  
  All workspaces and applications are now operational. We will continue to monitor the situation and make a more detailed update of what the underlying issue was.

• Resolution in progress | Jan 20, 2022 | 10:25 GMT+02:00

  The underlying issue has been identified and a fix is currently being applied. Once the fix has been deployed, Howspace will be back to normal.
  
  A more detailed report of the incident will be made after the issue has been resolved.
  
  Thank you for your patience.

• Investigating | Jan 20, 2022 | 10:00 GMT+02:00

  Howspace is experiencing an unexpected outage. The issue is being analysed and our team is taking steps to resolve it as soon as possible.
  
  The issue might cause some issues with accessing or updating workspaces.

A few bugs emerged after the planned maintenances last weekend.

Closed | Nov 29, 2021 | 15:33 GMT+02:00

There were problems with websocket connections causing auto-updating content in widgets and pages to fail. The issue was fixed 2021-11-29 9:10 (UTC+2).

There was an issue with uploading PDF files. The problem was fixed 2021-11-29 15:19 (UTC+2).

Affected components

• Howspace (in.howspace.com)

+Show history

• Open | Nov 29, 2021 | 06:00 GMT+02:00

  A few bugs emerged after the planned maintenances last weekend.
  
  There were problems with websocket connections causing auto-updating content in widgets and pages to fail. The issue was fixed this morning, 2021-11-29 9:10 (UTC+2).
  
  There is an issue with uploading PDF files. The problem is already fixed and fix will be deployed to production soon.
  

Unexpected outages during maintenance

Closed | Nov 29, 2021 | 13:56 GMT+02:00

All Howspace workspaces have been operational since the last outage on Saturday.

Outages were caused by a flaw in automated instance health monitoring in a load balancer. It mistakenly identified some service instances as not functional based on erroneous reasoning. This alone was not enough to
cause outages, but a new deployment happening at the same time caused the load balancer to deduce there was no healthy instances left to serve client requests.

The root cause is now fixed, and similar issues should not happen again.

Affected components

• Howspace (in.howspace.com)

+Show history

• Monitoring | Nov 26, 2021 | 20:20 GMT+02:00

  During planned maintenance, there were some unexpected complications. Total three outages occurred during the last 16 hours.
  
  All Howspace workspaces are operational again.
  
  We are now analyzing the root cause, and we are taking steps to prevent similar issues in the future.
  
  We will provide more information later.
  
  

Websocket connection issue

Closed | Sep 30, 2021 | 10:39 GMT+03:00

A brief connection issues occurred today at 10:39 - 10:50 where the connection between users and their workspaces was impaired. It was temporary due to an issue with server capacity.

The issue is being analyzed and we are taking steps to prevent similar issues in the future.

AWS migration maintenance downtime

Closed | Dec 12, 2020 | 19:40 GMT+02:00

Howspace should now be operational.

Affected components

• Howspace (in.howspace.com)

+Show history

• Resolved | Dec 12, 2020 | 19:38 GMT+02:00

  The maintenance break should be now over.
  Howspace will be operational for all, but it might take a few minutes for some individuals.

• Waiting | Dec 12, 2020 | 19:15 GMT+02:00

  It will still take some more time for Howspace to be operational.
  
  The next update will be at 6:15 pm EET.
  
  EDIT:
  Next update at 6:30 pm EET.
  
  EDIT2:
  The migration is progressing but will take some more time.
  Next update at 6:45 pm EET.
  
  EDIT3:
  Still waiting for the final steps of the migration.
  Next update at 7 pm EET.
  
  EDIT4:
  Still working on the migration.
  Next update at 7:15 pm EET.
  
  EDIT5:
  Still working on the migration.
  Next update at 7:30 pm EET.

• Resolution in progress | Dec 12, 2020 | 17:35 GMT+02:00

  There will be a short period of downtime related to the Amazon Web Service migration maintenance.
  
  The downtime should only take a short while, and we will update here as soon as Howspace is operational again, or at the latest at about 6 pm EET.