Web UI not showing output of my shell scripts

I have used Circle many times in the past and am setting up a new project. My project’s setup steps are in a single Ruby script called bin/setup. This script has $stdout.sync = true at the top which should force all output to be flushed immediately with no buffering.

And yet, the web UI does not show the output of my script. It does not show the output of scripts my script calls consistently either. This makes it extremely hard to debug and monitor what is going on and seems like a critical failure.

I have opened a support ticket, but am posting here in case I’m doing something wrong that others can help me with.

For context, I have been using this script daily for over a year without problems, running inside the exact same docker image I’ve set up with Circle. This leads me to believe it must be with how Circle is running the script or managing the output.

Hi @davetron5000! I couldn’t find any related issues that would cause this. Was Support able to help troubleshoot this?

They are working with me now. If you happen to know exactly how Circle executes commands in the container I could try replicating locally (since in this case, the container is based on my image, not circle’s).

1 Like

Hey @davetron5000! Thanks for your patience (I was sick for a few days). I hope Support has been able to help you, but if not, I’m happy to dig in. Do you have an example of what you’re trying to do?

I have a docker image on DockerHub and I’m trying to run a script inside that image. The Web UI has two failures: 1) it does not show the stdout of the script even though the script has been set to autoflush stdout and not buffer, 2) the script fails (because it doesn’t show stdout, I don’t know where it’s failing).

When I ssh into the box in Circle (rebuild with ssh), the script works fine and completes normally.

The script is a ruby script that is basically doing pretty normal stuff like bundle install and yarn install, and it’s for a pretty normal rails app.

I have used Circle CI since 1.0 and never had this particular problem. The only thing I can think is that whatever Circle is doing to run my script is somehow different from how I’m running it. Do you know what docker commands Circle uses to run scripts in the main container?

I wanted you to know I didn’t forget about this! I’ve been looking and asking around, but so far I have the same questions as the Support ticket, and I don’t want to make you repeat yourself. I’ll keep investigating :female_detective:

We’re having the same issue since start this week.
The commands we use to deploy to Google no longer works in the build script, but works when I ssh to the box and run the commands.
It’s very disruptive for us, I’ve filed a support ticket, waiting to hear back.
@davetron5000 If you find a solution please share, would help me out tremendously, thanks :slight_smile:

The support person asked me to put PAGER=cat into my environment and…that seemed to solve it. WTF.

I still have the ticket open to see if there is some explanation.

In my case, the script was getting stuck during Ruby on Rails database setup. Nothing in any part of it is interactive that I can tell, but putting PAGER=cat fixed it.

For those that don’t know what any of that means, PAGER is an environment variable that most UNIX systems look at to see what program they should use for showing output one page at a time. more or less are common programs that “page” through output. They require you to e.g. hit space to continue so you can read long output on the terminal.

cat is a program that just dumps its input to its output with no interactivity. So, if you used cat as your PAGER, you wouldn’t get any interactive page-at-a-time output. Thus, there must’ve been something somewhere that was piping output through more or less and waiting for spacebar. SUPER weird.