99.90%
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
MaintenanceScheduled Maintenance
CompletedThe scheduled maintenance has been completed.
In progressScheduled maintenance is currently in progress. We will provide updates as necessary.
ScheduledOn 19. 11. 2022 we will be updating components of our video processing pipeline. We do not expect any downtime to the Platform, but new videos will be transcoded later than usual during the maintenance window.
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
No incidents reported.
Slow responses from the the platform
ResolvedAt 14:01 UTC our applications lost connection to parts of its storage. While normally the connection should be automatically renewed, the automation failed to renew the connection. This led to errors when processing requests to the platform. At 14:05 UTC we identified the issue and manually triggered reconnects on appropriate parts of the platform, leading to recovery at 14:08 UTC. To prevent similar issues in the future, we’re adding more checks monitoring the connection between our platform and our storage, to detect and re-establish dropped connections. During the outage: * Running streams continued streaming * Some webinars speakers experienced losing connection to the webinar room * New viewers could not join webinars * Videos not cached by CDN were not be accessible * Admin interface of the TwentyThree platform was not accessible
ResolvedThis incident has been resolved.
MonitoringA fix has been implemented and we are monitoring the results.
InvestigatingWe are investigating an issue on the platform causing slow loading. We will update this thread when we know more.
No incidents reported.
No incidents reported.
No incidents reported.
Slow responses from the the platform
ResolvedThe issue was caused by an update to our infrastructure. The update rolled out at 9:31 UTC, and we started seeing failures at 9:33 UTC. At 9:38 UTC we reverted the changes on our infrastructure, leading to recovery with full recovery at 9:41 UTC. The rollout of the update prevented our applications from connecting to their storage, leading to errors when processing requests to the platform. This update had been tested before, and we did not find any issues during the test phase. We will review and improve the testing phase of similar updates, to prevent similar issues from ocurring in the future. During the incident: * Running webinars kept streaming, but new viewers could not join * Our admin interface was generally unvailable * Our API was generally unvailable
ResolvedThis incident has been resolved.
InvestigatingWe are investigating an issue on the platform causing slow loading. We will update this thread when we know more.
No incidents reported.
No incidents reported.
No incidents reported.
Issues connecting to a live streaming server
ResolvedOn 2. 11. 2022 between 12:35 and 12:37 UTC we experienced an Internet connectivity issue on our live streaming servers. This was caused by our infrastructure provider experiencing networking issues. This interrupted all active streams, which lost connection for the duration of the issue. Once the connectivity issue was resolved, streams recovered automatically.
No incidents reported.
No incidents reported.
Availability is (seconds up + seconds unmonitored) ÷ all seconds in the Europe/Copenhagen calendar month; only measured downtime counts against it, Maintenance included.
