CircleCI setup with testcontainers-go

Hello everyone,

We have a CircleCI test job that is executed each time when code is pushed to the remote branch. This test will start new machine using ubuntu image and start all golang test from repository. One of those tests requires redpanda container to verify the payload writen to Redpanda. In order to be able to start a Redpanda container, we defined all necessary container details using testcontainers-go library. I will copy/paste here the golang code which will start a redpanda docker container whenever test is executed:

func Redpanda(ctx context.Context, testCtx *test.Context) (testcontainers.Container, error) {
	name := fmt.Sprintf(
		"redpanda_%s_%d",
		testCtx.TestName(), time.Now().Unix(),
	)

	freePort, err := test.GetFreePort()
	if err != nil {
		return nil, fmt.Errorf("get free port: %w", err)
	}

	container, err := testcontainers.GenericContainer(
		ctx,
		testcontainers.GenericContainerRequest{
			ContainerRequest: testcontainers.ContainerRequest{
				Name:  name,
				Image: "redpandadata/redpanda:latest",
				ExposedPorts: []string{
					fmt.Sprintf("%d:9092/tcp", freePort),
					"9644/tcp",
				},
				Cmd: []string{
					"redpanda",
					"start",
					fmt.Sprintf("--advertise-kafka-addr=localhost:%d", freePort),
				},
				WaitingFor: wait.ForListeningPort(nat.Port(fmt.Sprintf("%d/tcp", 9092))).WithStartupTimeout(3*time.Minute),
			},
			Logger: testCtx.ServiceContext().ZapStdLogger(zapcore.DebugLevel),
		},
	)
	if err != nil {
		return nil, fmt.Errorf("create container: %w", err)
	}

	err = container.Start(ctx)
	if err != nil {
		return nil, fmt.Errorf("start container: %w", err)
	}

	port, err := container.MappedPort(ctx, "9092/tcp")
	if err != nil {
		return nil, fmt.Errorf("get mapped broker port: %w", err)
	}

	host, err := container.Host(ctx)
	if err != nil {
		return nil, fmt.Errorf("get host: %w", err)
	}

	broker := fmt.Sprintf("%s:%d", host, port.Int())

	testCtx.ServiceContext().SetValue("kafka.url", []string{broker})

	return container, nil
}

We have a problem with this setup since whenever we try to spin up redpanda container it will return the error ‘start container: context deadline exceeded’ and execution would fail while waiting for container to be ready. Does anyone else had a problem with similar setup? Any help would be welcome.

A complication is that “context deadline exceeded” just basically a golang level message meaning “something went wrong” and so does not provide any detail about what happened when you tried to start up the redpanda container.

A first step would be to emulate this within a shell (or config.yml) on the same system instance that you are trying to automate. That way you may see more detail in the logs regarding what is going wrong.

Also what size environment are you running this in and what else is running? Without details regarding the environment, it is hard to put forward any suggestions beyond checking that you have the spare resources to spin up the container.

Hey,

we were able to solve the problem so I will write the solution here for other people that came up with same issue. Apparently the problem was with the CircleCI ubuntu machine image that we used to spin up instance and run our tests as part of deployment. We updated image ( version that we used was 202104-01 ) which is outdated and set it to current. Now everything works fine.

Thanks for help!