Howspace (in.howspace.com) logo

Incident

Unexpected outage

Closed | Sep 28, 2022 | 15:15 GMT+03:00

The issue has been resolved and closed.

We update Howspace many times a week. Sometimes it also includes database
maintenance. These are normal everyday operations.

We had detected the database performance could be improved by altering an
index. With the change in this update, sorting would be more efficient, and each
Howspace page would load faster.

However, this time, the wrong order of operations during database maintenance
caused a premature deletion of a database index. The old index was dropped
before the build of the new one was completed. The system was unresponsive
because of excessive query times during the period needed for the new index to
rebuild.

There was not much we could do but wait for the index rebuild to finish. We
tried to prevent further slow queries by letting the database finish
indexing. Once the new index was in place, everything started to work normally
again. Now, based on some initial checks, the pages load faster.

We're very sorry for the downtime. We learned this could happen, and we will
implement precautions for the future.

Affected components

  • Howspace (in.howspace.com)

Resolved | Sep 28, 2022 | 10:41 GMT+03:00

Howspace services is fully operational again.

We will write and publish an incident report soon.

Investigating | Sep 28, 2022 | 10:21 GMT+03:00

Howspace workspaces are unavailable at the moment. The reason seems to be database re-indexing issue causing temporary failure to use the data.

We are investigating the situation.

Status Page powered by Admin Labs