Our account has 4 containers on CircleCI. We have a project (call it A) that is configured to use 3 containers and one (call it B) using a single container.
If a build of A is running and a build of B is requested, it’ll start in the spare container as you’d expect.
However, if a build of A is running and another build of A is queued, a build of B will wait behind A in the queue, despite there being a spare container that it could use straight away.
This really isn’t a major inconvenience, but it would be nice if it didn’t happen
We have the same issue.
We have project A using 4 containers and project B using 1.
We had 8 containers and decided going to 9 so that we can have at least one build of project B build anytime.
But if the next build in the queue is A and there’s not enough containers available for it, it will never build B even if there is 1 available container.
So now, we’ll probably go back to 8.
Is this on your roadmap? Do you think it makes sense?
I noticed the same thing this morning when doing some testing with multiple builds queued up. A single container build was waiting on a 4 container build to start, despite having capacity to fit it in and get started.
A common problem in managing a queue like this is ensuring that the large builds don’t get starved either. To take @sampart’s jobs from the first case (A with 3 containers, B with 1 container), if a lot of “B” jobs are getting queued up or are longer running, they can forever starve “A” jobs from starting.
Our situation is that each workflow run uses a number of 1 container jobs (e.g. setting up caches, standalone tools) and a few larger jobs that take multiple containers (e.g. test suites).
I think the best algorithm for us would be to prioritize the largest job that will fit in the open containers when something opens up, with a weight for how long something has been sitting in the queue to override it if needed.
I’d love the small jobs just to fill in the nooks and crannies that the larger jobs leave behind, vs. just having them delayed (and affecting items queued further beyond them).
CircleCI implements simple queuing for container usage. What this means is:
if a job is next up and there are insufficient containers available for its plan, it’ll queue
queued jobs prevent new jobs from running
job-containers are released when the container finishes, not when the job finishes (it did this in the past, but that was a bug)
It’s implemented this way because there are so many edge cases to cover to implement a priority queue. On an N container plan, if you have an N - 1 and 1 container project, a series of 1 container jobs could potentially block the N - 1 container jobs from running indefinitely, regardless of when the N - 1 container jobs were pushed.
We are addressing this in another way, though. We’ve been working on a new pricing model that charges based on container usage instead of the maximum number of containers you need in a month. In this model, you would pay for container credits which are consumed when one or more jobs run. This should eliminate plan-based job queueing.
We’re working on it now so that we’ll have new pricing models available in 2018. How early - I’m not sure yet but we’ll start putting notices out as soon as things firm up.
The first new plan(s) will be optional. If we did ever move into a migration, we’d give considerable lead time to all paying customers well before then.
Hi, I wonder if Circle CI is still pursuing the pricing model by container time instead of maximum number of containers? That would be a major motivation for my company to move from Team City into Circle CI.
