Unexpected outage

Closed | Sep 28, 2022 | 15:15 GMT+03:00

The issue has been resolved and closed.

We update Howspace many times a week. Sometimes it also includes database
maintenance. These are normal everyday operations.

We had detected the database performance could be improved by altering an
index. With the change in this update, sorting would be more efficient, and each
Howspace page would load faster.

However, this time, the wrong order of operations during database maintenance
caused a premature deletion of a database index. The old index was dropped
before the build of the new one was completed. The system was unresponsive
because of excessive query times during the period needed for the new index to
rebuild.

There was not much we could do but wait for the index rebuild to finish. We
tried to prevent further slow queries by letting the database finish
indexing. Once the new index was in place, everything started to work normally
again. Now, based on some initial checks, the pages load faster.

We're very sorry for the downtime. We learned this could happen, and we will
implement precautions for the future.

Affected components

• Howspace (in.howspace.com)

+Show history

• Resolved | Sep 28, 2022 | 10:41 GMT+03:00

  Howspace services is fully operational again.
  
  We will write and publish an incident report soon.
  

• Investigating | Sep 28, 2022 | 10:21 GMT+03:00

  Howspace workspaces are unavailable at the moment. The reason seems to be database re-indexing issue causing temporary failure to use the data.
  
  We are investigating the situation.
  
  

Temporary issue

Closed | Aug 26, 2022 | 14:11 GMT+03:00

The incident was swiftly resolved by our developer team once it was noticed. Steps have been taken to improve our internal alerts to prevent issues of this type in the future.

+Show history

• Resolved | Aug 12, 2022 | 09:21 GMT+03:00

  Steps have been taken that should resolve the issue. Howspace is up and running. We are monitoring the situation and will write and publish an incident report.

• Monitoring | Aug 12, 2022 | 08:10 GMT+03:00

  Steps have been taken that should resolve the issue. Howspace is up and running. We are monitoring the situation and will write and publish an incident report.

• Investigating | Aug 12, 2022 | 08:40 GMT+03:00

  The source of the issue has been identified. The problem is being actively worked on by our team.

• Open | Aug 12, 2022 | 07:55 GMT+03:00

  An issue has been identified that interferes with access to Howspace. Investigation into the issue has begun and our team is working on resolving the issue .

Single sign-on feature not available

Closed | Feb 01, 2022 | 00:28 GMT+02:00

During planned maintenance, there were unexpected complications, which made the Single sign-on feature unavailable to some customers. After noticing the issue, corrective actions were made.

We apologize for any inconvenience it might have caused and thank you for your patience and understanding.

Unexpected outage

Closed | Jan 20, 2022 | 14:31 GMT+02:00

The issue has been resolved and closed.

An update that went to production during the morning (2022-01-20) included a new feature which under a heavy load caused database performance issues. It cascaded and caused performance issues which appeared to users as 504 Gateway Timeout or as a websocket connection error.

When the issue was identified actions were taken to resolve it as soon as possible.

We apologise for any inconvenience it might have caused and thank you for your patience and understanding.

+Show history

• Resolved | Jan 20, 2022 | 10:46 GMT+02:00

  Howspace is back online. We have applied a fix that resolved the issue.
  
  All workspaces and applications are now operational. We will continue to monitor the situation and make a more detailed update of what the underlying issue was.

• Resolution in progress | Jan 20, 2022 | 10:25 GMT+02:00

  The underlying issue has been identified and a fix is currently being applied. Once the fix has been deployed, Howspace will be back to normal.
  
  A more detailed report of the incident will be made after the issue has been resolved.
  
  Thank you for your patience.

• Investigating | Jan 20, 2022 | 10:00 GMT+02:00

  Howspace is experiencing an unexpected outage. The issue is being analysed and our team is taking steps to resolve it as soon as possible.
  
  The issue might cause some issues with accessing or updating workspaces.

A few bugs emerged after the planned maintenances last weekend.

Closed | Nov 29, 2021 | 15:33 GMT+02:00

There were problems with websocket connections causing auto-updating content in widgets and pages to fail. The issue was fixed 2021-11-29 9:10 (UTC+2).

There was an issue with uploading PDF files. The problem was fixed 2021-11-29 15:19 (UTC+2).

Affected components

• Howspace (in.howspace.com)

+Show history

• Open | Nov 29, 2021 | 06:00 GMT+02:00

  A few bugs emerged after the planned maintenances last weekend.
  
  There were problems with websocket connections causing auto-updating content in widgets and pages to fail. The issue was fixed this morning, 2021-11-29 9:10 (UTC+2).
  
  There is an issue with uploading PDF files. The problem is already fixed and fix will be deployed to production soon.