No incidents reported
15th June 2021, 13:00
We have detected on-going issues in our Chicago location regarding high-latency and packet loss and are investigating.
15th June 2021, 15:30
A notice has been posted by our upstream, INAP, regarding the on-going issues.
"This is to notify you the INAP NOC is aware of a network issue in the CHG POP. NOC engineers are currently investigating and we will continue to update you as we receive more information." INAP's "Operational Transparency" page can be found here: https://ot.inap.com/
15th June 2021, 16:10
A notice has been posted by our upstream, INAP, regarding the on-going issues.
"The INAP NOC and Senior Network Engineers are currently engaged mitigating a large DDoS impacting multiple customers out of the Chicago Market. Some early mitigation steps were successful, but had to be adjusted as the attack changed. Therefore customers may have seen periods of resolution before losing connection again. At this time we have isolated the attack and connectivity should be restoring. We sincerely apologize for the connection issues and impact to your business. We are putting additional measures in place to ensure stability in the CHI Market. An RFO of the events will be issued within 3 business days. "
16th June 2021, 9:40
A notice has been posted by our upstream, INAP, regarding the on-going issues.
"Several attacks were mitigated throughout the night. Customers may have seen periods of latency, packet loss and time-outs during the events. We are continuing to monitor the situation closely. The last event was at 03:46 CDT. Please reach out to INAP support with any questions or concerns."
17th June 2021, 1:31
A notice has been posted by our upstream, INAP, regarding the on-going issues.
"We are currently mitigating several attacks. All Engineering teams are engaged as well as senior teams to resolve the current threats. Customers may see periods of latency, packet loss and time-outs during the event"
17th June 2021, 14:00
A notice has been posted by our upstream, INAP, regarding the on-going issues.
"We are currently mitigating another attack. All Engineering teams are engaged as well as senior teams to resolve the current threats. Customers may see periods of latency, packet loss and time-outs during the event. Please reach out to INAP support with any questions or concerns. We appreciate your patience and understanding." "Hello the last incident event was mitigated at 10:44 CDT. There is no current customer impact. Senior Network teams are actively working on a long term solution. Please reach out to INAP support with any questions or concerns."
17th June 2021, 16:00
It would appear from our monitoring that the issues have been resolved with INAP. If they share any other updates, we will post here.
18th June 2021, 15:50
A notice has been posted by our upstream, INAP, regarding the on-going issues.
"There have been no further incidents since the last update. We are continuing to monitor the situation closely. Please reach out to INAP support with any questions or concerns."
No incidents reported
No incidents reported
5th June 2021, 21:00
We are currently planning on doing full location maintenance in Dallas, Texas on12th June 2021. This date is flexible and you will be updated if something changes. We will be upgrading networking systems and moving our systems into a different set of racks. Expected downtime is hopefully AT MAX 8 hours long, starting at 10AM CST. This downtime could be longer.
If you have any questions please contact us through our ticketing system. We hope to have everyone back online by 6PM CST.
If you are wondering "Why are they doing this on a Saturday?" it is because this is a two person job and we are having to call in some extra set of hands, and they were only available on Saturday. Also we are doing something we never have before! We're going to be live-streaming the move to everyone if you would like to watch. Details will be posted on Saturday.
12th June 2021, 10:00
Dallas Maintenance is starting. If you're interested in the live stream we will share a link in Discord.
12th June 2021, 11:30
We ran into some trouble regarding the post positions in our new racks, this means there's a delay. We're currently repositioning them. Should have it sorted in around 30 minutes. We hope to start moving servers by 1pm. We'll be moving them 5 at a time, tops 15-20 minutes downtime for each node.
13th June 2021, 00:30
Dallas Maintenance has now finished! We have migrated to our new racks successfully and upgraded our network.
We have a time-lapse from the Twitch stream we did showing the migration in the discord for those interested.