The traffic has normalized and the backlog has cleared out. Please let us know if you're continuing to see issues.
Posted Feb 26, 2024 - 21:38 UTC
Update
The team did some testing and felt confident they had a safe way to increase capacity. They were able to double the throughput of production. The process is first in, first out so with patience you're jobs should be finishing up within the next hour or so.
The bottleneck is on Syndicate itself so it impacts publishing as well as uploading.
Posted Feb 26, 2024 - 20:08 UTC
Monitoring
Engineering has evaluated a plan to temporarily increase capacity to clear the backlog and there is a risk that it would cancel all publishing jobs, For the moment, we've decided to just monitor the queue to ensure documents continue to be published.
Please be patient, and your outputs should be published. Our current calculation is the last one in the queue should be completed no later than 5:00 p.m. EST.
If you'd like to cancel your publications and try again later, you can select Cancel next to the job on the publications page.
Posted Feb 26, 2024 - 19:28 UTC
Identified
We are aware of an issue impacting publishing to Syndicate. Jobs are processing but there is an unexpected spike in publishing volume creating a backlog. We're reviewing the situation to determine if we can speed up the flow. All jobs should process successfully but it will take much longer than normal.
Posted Feb 26, 2024 - 18:58 UTC
This incident affected: Syndicate - US Hosting (Syndicate (CDS)) and Create - US Hosting (Studio Publishing Service).