Monitoring - Access to gitlab.rc.colorado.edu has been restored, and configuration changes have been made. We are monitoring to observe whether this is sufficient to keep the service active.

If your account is still locked, please follow the unlock instructions that were sent to you in email.
Mar 10, 14:27 MST
Update - Our security office has reviewed our logs and confirmed our understanding of the attack. It does seem as though there was no actual unauthorized access, aside from the fact that the attacker does seem to have a list of at least some valid accounts for the GitLab server. We are investigating how this list may have been obtained, but it likely has to do with intentionally public access to projects stored in the server.

We are about to restore access to this server, and make two changes:

- New accounts will require admin approval in order to be created. (This should be handled using an existing automated workflow.)

- Unauthenticated connections will be rate-limited by IP address.

These changes--particularly the rate limiting--may require some tuning, so we will make an initial estimate and adjust if necessary.
Mar 10, 14:19 MST
Update - We have upgraded all packages relevant to this Gitlab instance, but the attack is ongoing. To protect the server, and its account credentials, we are going to leave the server offline while the security office reviews our logs and recommends a mitigation plan.
Mar 9, 12:11 MST
Investigating - We are investigating an apparently ongoing attack on our externally-facing GitLab server. Details about the attack have been communicated with the IT security office. We are taking this opportunity to upgrade the OS and GitLab software to ensure we have all the latest security updates. So far we have no indication of any actual unauthorized access or compromise.

You may have received a notification that your account was locked, with "Unlock instructions." Until this attack has been completely addressed, we recommend _not_ attempting to unlock or access your GitLab account.

More information will be provided here as it is available.
Mar 9, 10:41 MST
Research Computing Core ? Operational
Science Network ? Operational
RMACC Summit ? Operational
Blanca ? Operational
PetaLibrary ? Operational
EnginFrame ? Operational
JupyterHub ? Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Apr 21, 2021

No incidents reported today.

Apr 20, 2021

No incidents reported.

Apr 19, 2021

No incidents reported.

Apr 18, 2021

No incidents reported.

Apr 17, 2021

No incidents reported.

Apr 16, 2021

No incidents reported.

Apr 15, 2021

No incidents reported.

Apr 14, 2021

No incidents reported.

Apr 13, 2021

No incidents reported.

Apr 12, 2021

No incidents reported.

Apr 11, 2021

No incidents reported.

Apr 10, 2021

No incidents reported.

Apr 9, 2021

No incidents reported.

Apr 8, 2021

No incidents reported.

Apr 7, 2021

No incidents reported.