Skip to content

Service recovery Humanities IT Services

Due to a fire in our main datacenter, most of the services hosted by Humanities IT Services are currently down.

For information on the status of other UU services, please visit the liveblog page on uu.nl.

What are we working on

We're working on our plan for restoring full services once our datacenter is back online. We've reached out to our service owners to figure out which services need to be prioritized.

In addition, we're handling some urgent cases to restore additional emergency services.

The future

We are still waiting for power to be restored to the datacenter, which should be on Wednesday. Once we have power, we (and ITS) will need to verify that our hardware is working properly and then begin working on restoring our core services.

We're still dealing with a number of unknowns, so we are hesitant to provide any kind of timeline. Once we've restored initial core services, we'll be able to provide a more detailed timeline.

Contact

Matching/Osiris

We've received a few mails about the status of the matching forms. Please contact your student desk (studiepunt) about this; We cannot provide support for the matching forms/Osiris.

Humanities IT Services

If you have any questions or concerns, please contact us at our regular email address. Please note that our team is busy, so we may not be able to respond to your email immediately.

Timeline

  • 1 min read

Initial timeline

We've created an initial timeline for restoring services in our datacenter.

Tomorrow, Wednesday, NorthC will restore power to the datacenter, after which ITS will need to start up their infrastructure. They have indicated that they expect to start doing so after 17:00.

Hence, we plan to start up our infrastructure on Thursday, May 14th. We will initially focus on our own core infrastructure before moving on to other services. We've been in contact with our stakeholders and have made an initial prioritization list. With any luck, those will be online by the end of Friday. Full restoration may take a few more days after that.

Important notes

  1. This timeline is dependent on no major issues arising during restoration. If that does happen, we will communicate this on this website.
  2. During restoration of core infrastructure, we won't be able to make changes to any now-running services.
  3. Some services may appear to start working; we kindly ask you to wait until we've confirmed that they are fully operational before using them.-
  • 1 min read

DNS issues resolved and landing pages up

ITS has managed to resolve the DNS issue, which allows us to bring up several emergency landing pages and other services.

Most sites in our wordpress environments should now be redirected to this page, while some others will be directed to dedicated info pages for those sites.

  • 1 min read

Monday recap

We've been able to get the first few emergency pages up and running (most importantly, diergeneeskunde.nl) and deployed servers for other critical services.

However, we're still blocked by a DNS issue. We hope that ITS will be able to resolve it by tomorrow morning at the latest.

  • 1 min read

Monday morning update

We're currently hard at work coordinating our recovery plan internally and with ITS; at this point it seems likely that we won't be able to start full recovery until Thursday at the earliest. This is due to delays in restoring power at the datacenter.

In the meantime we're also working on deploying temporary infrastructure to get critical services back up, as well as a general status page for the UU wordpress environment.

However, we're currently blocked by a DNS issue that ITS will have to resolve before we can proceed with bringing the temporary infrastructure online.

  • 1 min read

Expected delay

We've received word that NorthC is not going to be able to restore power to our equipment tomorrow.

We will evaluate this news tomorrow and see what can be done in the meantime.

  • 1 min read

Final update for today

NorthC has indicated that it may take 72 hours to restore power. This means that we may be able to start bringing up infrastructure again on Monday at the earliest. However, full restoration will take longer.

As such, we still cannot give a concrete timeline at this point. We hope to have a more detailed update on Monday.

In the meantime, we have been unable to get the temporary landing page running due to unforseen issues outside our control. We're hoping to get this resolved soon, but we can make no promises at this point.

  • 1 min read

Initial site visit by ITS

We have received word from ITS that they've visited the datacenter. While the building is still without power, our hardware does not appear to be affected by the fire.

We are still waiting on NorthC to provide us with a timeline for when power is restored and when we can start starting our servers.

  • 1 min read

Restoring critical services

We have started working on restoring critical (internal) services in our alternate datacenter.

This will include a temporary landing page for users informing them of the outage.

  • 1 min read

Fire under control

The fire department has declared 'brand meester', which means the fire is under control and is no longer a risk of spreading to other sections of the building.

  • 1 min read

Backups verified

We have confirmed that our off-site backups are present and accounted for. We expect little to no data-loss to occur once our servers are restored.