Builds stuck in Queue?

queue

#1

We are getting this message and all our builds are now stuck.

CircleCI has spent 07:02 acquiring containers for this build. We’re sorry; this is our fault. Typically you should only see this when load spikes overwhelm our auto-scaling; waiting to acquire containers should be brief and infrequent. This build was queued behind the following builds for 00:00

Anyone else seen this?


#2

We are having similar issues, out of 10 builds 9 are in queue for 10minutes now and just 1 passed (and failed with a cryptic message that I’m not sure if is my fault since I did change near that line)


#3

Hi,

I apologize for the downtime. One backend service updated and blocked 2.0 builds because of an API incompatibility. Our engineers resolved this and 2.0 builds are running again.

Builds that were stuck in the queue during this time will have to be retried in order to run.


#4

Did the circle.yml format changed as well?

For 2 hours, all our builds are failing with Unexpected environment preparation error: json: error calling MarshalJSON for type config.StepDescription: json: unsupported type: map[interface {}]interface {}. We didn’t touch the circle.yml so I’m guessing the parser must have been updated or something.


#5

No. It seems like pieces of the platform were down. Builds are running again. The config format has not changed.


#6

Hum… Any idea why this error would appear in the Spin up Environment? I’m not sure what is config.StepDescription.

Working build:

Non working build:

The circle.yml is the same in both cases. The 2 builds were 20 min apart, 4 hours ago. And now, no build work.


#7

We are also getting the same thing Unexpected environment preparation error: json: error calling MarshalJSON for type config.StepDescription: json: unsupported type: map[interface {}]interface {}

No changes in our circle.yml


#8

@billy @mathieudutour Thanks for letting us know! I reached out to our team and they said a commit was pushed (and now reverted) that impacted a small subset of builds. I was not able to reproduce it personally. The fix will be out ASAP.


#9

We were getting the same error a few days ago:

Unexpected environment preparation error: json: error calling MarshalJSON for type config.StepDescription: json: unsupported type: map[interface {}]interface {}

It was fixed late Friday night (UTC), but the error seems to have returned. All of our builds are failing. Any chance you could take a look?


#10

@SamirTalwar You need to remove the - from your env variables to look like what I have below; it will resolve that error.

      - type: shell
        name: 'Run tests'
        command: npm test
        environment:
          FOO: bar

#11

@rohara Well spotted! That was weird. Thanks for finding it for us!


#12

Just wanted to close the loop on this. The issues with the beta fleet were resolved and all builds should be flowing through.

Please open a new topic for any new issues!


#13

Hi, I am getting the same issue and cannot run my tests with beta version:

CircleCI has spent 02:43 acquiring containers for this build. We’re sorry; this is our fault. Typically you should only see this when load spikes overwhelm our auto-scaling; waiting to acquire containers should be brief and infrequent.

I am new to beta version. Any updates since last week?


#14

That means your config is invalid. If you want to PM me your build link, I can take a look at what is wrong.


#15

Also experiencing issues with queueing, except our container is queued for 15:51:16 D=. I’ve PM’d you the link as well.


#16

A common thing to look for when you have endless queueing is how you’re setting your env vars. Change your = to : or vise versa; the consistency of circle.yml is not yet where we need it.


#17

All of our 2.0 builds are currently stuck, although 1.0 builds are going through.


#18

@rfay Based on your post here, I assume it was a syntax error and you have running builds now. If not, please direct message me with a link to builds that are showing “Queued” for long periods.


#19

Thanks - yes our builds are now running. It wasn’t just this, somebody kindly must have intervened because I sorted this out well before our builds got unstuck.


#20

I think I’m seeing a similar issue. Environments are taking about 6 minutes to spin up and sometimes they timeout after about 10.