Different shell output when running in CircleCI vs local


#1

Hi,

I have a shell script that yield different output when running locally vs running it on CircleCI.
The script runs just fine in CircleCI, it is just the output that bothers me (it’s important for debugging)

There are two main differences when running in CircleCI:

  1. The output is printed twice
  2. The grep command output is not printed at all.

Here is the script:

#!/bin/bash

set -x

#We assume that rabbitmq and pse services are already running (check out circleci.yml for that)

#creating dummy queue (test-pse-q) and bind it to event-to-index exchange
curl --fail -S -i -u admin:admin -H "content-type:application/json" -XPUT -d'{"vhost":"/","name":"test-pse-q","durable":"true","auto_delete":"false","arguments":{}}' http://localhost:15673/api/queues/%2F/test-pse-q
curl --fail -S -i -u admin:admin -H "content-type:application/json" -XPOST -d'{"vhost":"/","source":"events-to-index","destination_type":"q","destination":"test-pse-q","routing_key":"event.#","arguments":{}}' http://localhost:15673/api/bindings/%2F/e/events-to-index/q/test-pse-q

#send message to events-to-enrich exchange
curl --fail -S -i -u admin:admin -H "content-type:application/json" -XPOST -d'{"vhost":"/","name":"events-to-enrich","properties":{"delivery_mode":2,"headers":{}},"routing_key":"event.testme","delivery_mode":"2","payload":"{\"key\": \"value\"}","headers":{},"props":{},"payload_encoding":"string"}' http://localhost:15673/api/exchanges/%2f/events-to-enrich/publish

#test that the message reached the dummy queue with the enrichment of gaia section
curl --fail -S -i -u admin:admin -H "content-type:application/json" -XPOST -d'{"vhost":"/","name":"test-pse-q","truncate":"50000","requeue":"true","encoding":"auto","count":"1"}' http://localhost:15673/api/queues/%2F/test-pse-q/get | grep gaia | grep incoming_time

Here is the output when running it locally:

core@core-01 ~/pre-store-enricher $ ./SystemTest.sh
+ curl --fail -S -i -u myuser:mypass -H content-type:application/json -XPUT '-d{"vhost":"/","name":"test-pse-q","durable":"true","auto_delete":"false","arguments":{}}' http://localhost:15672/api/queues/%2F/test-pse-q
HTTP/1.1 204 No Content
Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact)
Date: Mon, 04 Apr 2016 10:38:09 GMT
Content-Type: application/json
Content-Length: 0

+ curl --fail -S -i -u myuser:mypass -H content-type:application/json -XPOST '-d{"vhost":"/","source":"events-to-index","destination_type":"q","destination":"test-pse-q","routing_key":"event.#","arguments":{}}' http://localhost:15672/api/bindings/%2F/e/events-to-index/q/test-pse-q
HTTP/1.1 201 Created
Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact)
Location: test-pse-q/event.%2523
Date: Mon, 04 Apr 2016 10:38:09 GMT
Content-Type: application/json
Content-Length: 0

+ curl --fail -S -i -u myuser:mypass -H content-type:application/json -XPOST '-d{"vhost":"/","name":"events-to-enrich","properties":{"delivery_mode":2,"headers":{}},"routing_key":"event.testme","delivery_mode":"2","payload":"{\"key\": \"value\"}","headers":{},"props":{},"payload_encoding":"string"}' http://localhost:15672/api/exchanges/%2f/events-to-enrich/publish
HTTP/1.1 200 OK
Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact)
Date: Mon, 04 Apr 2016 10:38:09 GMT
Content-Type: application/json
Content-Length: 15
Cache-Control: no-cache

{"routed":true}+ grep incoming_time
+ grep gaia
+ curl --fail -S -i -u myuser:mypass -H content-type:application/json -XPOST '-d{"vhost":"/","name":"test-pse-q","truncate":"50000","requeue":"true","encoding":"auto","count":"1"}' http://localhost:15672/api/queues/%2F/test-pse-q/get
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   397  100   298  100    99  10723   3562 --:--:-- --:--:-- --:--:-- 11461
[{"payload_bytes":63,"redelivered":true,"exchange":"events-to-index","routing_key":"event.testme","message_count":3,"properties":{"delivery_mode":2,"content_type":"application/json"},"payload":"{\"gaia\":{\"incoming_time\":\"2016-04-03T12:42:40Z\"},\"key\":\"value\"}","payload_encoding":"string"}]

Here is the output from CircleCI:


./SystemTest.sh
+ curl --fail -S -i -u admin:admin -H content-type:application/json -XPUT '-d{"vhost":"/","name":"test-pse-q","durable":"true","auto_delete":"false","arguments":{}}' http://localhost:15673/api/queues/%2F/test-pse-q
HTTP/1.1 204 No Content

Server: MochiWeb/1.1 WebMachine/1.10.7 (we're aiming for the pool, right?)

Date: Mon, 04 Apr 2016 10:21:03 GMT

Content-Type: application/json

Content-Length: 0



+ curl --fail -S -i -u admin:admin -H content-type:application/json -XPOST '-d{"vhost":"/","source":"events-to-index","destination_type":"q","destination":"test-pse-q","routing_key":"event.#","arguments":{}}' http://localhost:15673/api/bindings/%2F/e/events-to-index/q/test-pse-q
HTTP/1.1 201 Created

Server: MochiWeb/1.1 WebMachine/1.10.7 (we're aiming for the pool, right?)

Location: test-pse-q/event.%2523

Date: Mon, 04 Apr 2016 10:21:03 GMT

Content-Type: application/json

Content-Length: 0



+ curl --fail -S -i -u admin:admin -H content-type:application/json -XPOST '-d{"vhost":"/","name":"events-to-enrich","properties":{"delivery_mode":2,"headers":{}},"routing_key":"event.testme","delivery_mode":"2","payload":"{\"key\": \"value\"}","headers":{},"props":{},"payload_encoding":"string"}' http://localhost:15673/api/exchanges/%2f/events-to-enrich/publish
HTTP/1.1 200 OK

Server: MochiWeb/1.1 WebMachine/1.10.7 (we're aiming for the pool, right?)

Date: Mon, 04 Apr 2016 10:21:03 GMT

Content-Type: application/json

Content-Length: 15

Cache-Control: no-cache



{"routed":true}+ grep incoming_time
+ grep gaia
+ curl --fail -S -i -u admin:admin -H content-type:application/json -XPOST '-d{"vhost":"/","name":"test-pse-q","truncate":"50000","requeue":"true","encoding":"auto","count":"1"}' http://localhost:15673/api/queues/%2F/test-pse-q/get
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

100   398  100   299  100    99  50235  16633 --:--:-- --:--:-- --:--:-- 59800
+ curl --fail -S -i -u admin:admin -H content-type:application/json -XPUT '-d{"vhost":"/","name":"test-pse-q","durable":"true","auto_delete":"false","arguments":{}}' http://localhost:15673/api/queues/%2F/test-pse-q
HTTP/1.1 204 No Content

Server: MochiWeb/1.1 WebMachine/1.10.7 (we're aiming for the pool, right?)

Date: Mon, 04 Apr 2016 10:21:03 GMT

Content-Type: application/json

Content-Length: 0



+ curl --fail -S -i -u admin:admin -H content-type:application/json -XPOST '-d{"vhost":"/","source":"events-to-index","destination_type":"q","destination":"test-pse-q","routing_key":"event.#","arguments":{}}' http://localhost:15673/api/bindings/%2F/e/events-to-index/q/test-pse-q
HTTP/1.1 201 Created

Server: MochiWeb/1.1 WebMachine/1.10.7 (we're aiming for the pool, right?)

Location: test-pse-q/event.%2523

Date: Mon, 04 Apr 2016 10:21:03 GMT

Content-Type: application/json

Content-Length: 0



+ curl --fail -S -i -u admin:admin -H content-type:application/json -XPOST '-d{"vhost":"/","name":"events-to-enrich","properties":{"delivery_mode":2,"headers":{}},"routing_key":"event.testme","delivery_mode":"2","payload":"{\"key\": \"value\"}","headers":{},"props":{},"payload_encoding":"string"}' http://localhost:15673/api/exchanges/%2f/events-to-enrich/publish
HTTP/1.1 200 OK

Server: MochiWeb/1.1 WebMachine/1.10.7 (we're aiming for the pool, right?)

Date: Mon, 04 Apr 2016 10:21:03 GMT

Content-Type: application/json

Content-Length: 15

Cache-Control: no-cache



{"routed":true}+ grep incoming_time
+ grep gaia
+ curl --fail -S -i -u admin:admin -H content-type:application/json -XPOST '-d{"vhost":"/","name":"test-pse-q","truncate":"50000","requeue":"true","encoding":"auto","count":"1"}' http://localhost:15673/api/queues/%2F/test-pse-q/get
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

100   398  100   299  100    99  50235  16633 --:--:-- --:--:-- --:--:-- 59800

Thanks,
Shay


#2