We’ve recreated the issue loom was seeing and have released a fix. We’ll renable the org shortly.
Please don’t hesitate to reach out again if you spot anything else.
Thanks again
Dom
We’ve seen consistent failures in the past 2 hours. The first failure is when the v2 container runtime is being used. Could you opt out our org again?
Examples:
github/loomhq/loom/180705/workflows/dae953e6-f14d-465a-8ede-781fed76e6db/jobs/2623678
github/loomhq/loom/180705/workflows/dae953e6-f14d-465a-8ede-781fed76e6db/jobs/2623671
github/loomhq/loom/180705/workflows/dae953e6-f14d-465a-8ede-781fed76e6db/jobs/2623671
Sorry about that @victorlee134
I’ve run the opt out, it can take around 10 minutes to apply.
Just a heads up: The first linked issue did start to occur on the old infrastructure so you may still see that one. See: loomhq/loom/180697/workflows/93f2900c-0d84-4700-9828-d45e991d8cf9/jobs/2623493
Thanks @DominicLavery!
And yeah my mistake, pasted the wrong pipeline failure link
We believe the issue attentive-mobile saw was due to a new pid limit that was being automatically & implicitly applied in the new environment.
We’ve increased limit substantially and will opt the org back in. I’ll keep an eye on the builds to make sure that the issue is fixed and reapply the opt out if it re-occurs
Thanks
Thank you @DominicLavery for the update! We will watch the builds too and let you know if we see anything unusual 
1 Like
Hey @DominicLavery ,
It’s happening again: https://app.circleci.com/pipelines/github/attentive-mobile/code/563236/workflows/9d8218d9-3e4e-4282-ad26-05f1deebdf32/jobs/7257268/parallel-runs/0/steps/0-122?invite=true#step-122-639983_99
It’s using v2 container runtime.
Could you opt us out again?
Thanks!
Bian
Another question, is there a way we could enable it on an individual job so that we can test it out before it’s rolled out to our org?
Hey @masterginger
Sorry to hear that! I’ve reapplied the opt out.
We don’t currently have the ability to run it on a per job/branch level. We can do it on a project level if that could be of any help?
Thanks
Dom
Hey @DominicLavery ,
Thanks for the quick response! We’ve created a test project https://app.circleci.com/pipelines/github/attentive-mobile/v2-container-runtime-test , could you enable v2 container runtime for that project only?
We will put the reproducible code and steps in that project and hopefully that will provide good signals for CircleCI to debug the issue
Thanks,
Bian
Thanks @masterginger! That is really helpful.
I’ve opted the new project in and have the next fix lined up. That should hopefully go live on Monday .