Can't connect to node HTTP server running on localhost


#1

I’m trying to write an integration test which for now simply launches a Node server, then queries it to check if it’s running properly. The test works locally, and on CircleCI I was also apparently able to launch the Node server on port 9000. However, my tests fail with net::ERR_CONNECTION_REFUSED. Would you have any idea?

Test looks like this:

const puppeteer = require('puppeteer')

let browser

beforeAll(async () => {
  browser = await puppeteer.launch()
})

test('index', async () => {
  const page = await browser.newPage()
  const siteUrl = 'http://localhost:9000' // process.env.DEPLOY_URL || 'http://localhost:8000'
  console.log('Testing site URL:', siteUrl)
  await page.goto(siteUrl, {waitUntil: 'networkidle2'})
  const innerText = await page.evaluate(sel => document.querySelector(sel).innerText, '[data-test="description"] > span')

  expect(innerText).toEqual('Explore meditation with leading authorities on wisdom, compassion, and mindfulness')
})

afterAll(async () => {
  await browser.close()
})

And CircleCI config looks like this:

# Javascript Node CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-javascript/ for more details
#
version: 2
jobs:
  build:
    docker:
      # specify the version you desire here
      - image: circleci/node:9.4-browsers

      # Specify service dependencies here if necessary
      # CircleCI maintains a library of pre-built images
      # documented at https://circleci.com/docs/2.0/circleci-images/
      # - image: circleci/mongo:3.4.4

    working_directory: ~/repo

    steps:
      - checkout

      # Download and cache dependencies
      - restore_cache:
          keys:
          - v1-dependencies-{{ checksum "package.json" }}
          # fallback to using the latest cache if no exact match is found
          - v1-dependencies-

      - run: yarn install

      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}

      - run: yarn build

      - run: mkdir logs

      # start Node server
      - run: sh -c "yarn start | tee logs/node_server.log" & PID=$! && echo "PID is $PID and PWD is $PWD" && sleep 30

      - store_artifacts:
          path: logs

      # run tests!
      - run: yarn test

yarn test is just executing jest test/.

Thanks very much for any help :slightly_smiling_face:


#2

@levlaz would you have an idea, by any chance? :slightly_smiling_face:


#3

I’m guessing that your tests are starting too quickly, before your HTTP server has started up. The solution may be as simple as sleep 5 && yarn test, or you can do something a bit fancier that checks if the socket has started up.

To start with, you could just let the build fail with the SSH option, then get a remote console and run the tests (I guess cd ~/repo && yarn test?). By the time you get a connection, your HTTP server should definitely be started.


#4

@halfer there is already a sleep of 30 seconds (see config and screenshot above). When following the build through the CircleCI console I see that the HTTP server is fully started after less than one second, then it idly waits for 30 seconds. But then when yarn test kicks in, it fails nevertheless. It also fails if I replace yarn test with curl http://localhost:9000.

However, yarn test and curl both work fine when I SSH into the machine then run manually.

So it’s just when CircleCI itself is running the tests, it can’t resolve localhost for some reason. Is it somehow related to docker (which I know nothing about)?


#5

Hmm, OK. I am not familiar with Yarn, but it looks to me that your server is called Gatsby, and that seems to be starting up after the sleep, thus defeating its purpose. Moreover, I cannot see where gatsby is started up in your config.

I don’t think so. The run line is run inside your container, and if you can run the tests in a subsequent SSH session, then you’ve not changed anything.

I wonder, try using 127.0.0.1 instead of localhost? I have seen some rare envs where the name was not defined.


#6

I have configured my package.json with this:

  "scripts": {
    "build": "gatsby build",
    "start": "gatsby serve",
    ...
  }

So first the build is run, which produces static HTML/CSS/JS, then gatsby serve just spins an HTTP server serving those assets from localhost:9000. As the screenshot above shows, that command starts fine: “Running on port 9000”.


#8

OK, 127.0.0.1 didn’t seem to work either, but I could detect the IP with node -p "require('ip').address()". So problem is fixed with the config below. It’s still really puzzling that localhost doesn’t work, that looks like a big limitation.

# Javascript Node CircleCI 2.0 configuration file
#
# Check https://circleci.com/docs/2.0/language-javascript/ for more details
version: 2
jobs:
  build:
    docker:
      # specify the version you desire here
      - image: circleci/node:9.5-browsers

      # Specify other service dependencies here if necessary
      # CircleCI maintains a library of pre-built images
      # documented at https://circleci.com/docs/2.0/circleci-images/

    working_directory: ~/repo

    steps:
      - checkout

      # Download and cache dependencies
      - restore_cache:
          keys:
          - v1-dependencies-{{ checksum "package.json" }}
          # fallback to using the latest cache if no exact match is found
          - v1-dependencies-

      - run: yarn install

      # workaround for failing node-gyp build of native dep gatsby>sharp>libvips.
      # Remove when https://github.com/yarnpkg/yarn/issues/5152 is fixed.
      - run:
          name: Rebuild native dependencies
          command: npm rebuild
          when: on_fail

      - save_cache:
          paths:
            - node_modules
          key: v1-dependencies-{{ checksum "package.json" }}

      - run: yarn build

      - run:
          name: starting node server
          command: mkdir logs && yarn start 2>&1 | tee logs/node_server.log
          background: true

      - run:
          name: storing local IP address
          command: node -p "require('ip').address()" > logs/ip.log

      - run:
          name: waiting for node server to start
          command: wget --retry-connrefused --waitretry=2 -t 5 "http://$(cat logs/ip.log):9000" > /dev/null

      - run:
          name: running tests
          command: yarn test

      - store_artifacts:
          path: logs

And changed the test with const siteUrl =http://${require(‘ip’).address()}:9000``.

(NB: the documentation mentions a config run: echo 127.0.0.1 devhost | sudo tee -a /etc/hosts, but with no explanation on what that would be for (and I didn’t try adding that).)


#9

How big is the limitation if you solved the problem? :grinning:

I’m not sure what issue you’re running into here, but hey, it’s fixed now. For anything requiring more than a single repo, I tend to install Docker Compose, and so my HTTP/WS operations are local names on a private Docker LAN anyway. If you ever get stuck again, it may be worth a go, since DC can be run/debugged locally too.


#10

I just mean that localhost is a standard Unix feature, it should « just work » for any newcomer figuring out their first circleCI setup. If not, it should be documented properly. But anyway, thanks a lot for your help.

Activé sam., févr. 10, 2018 à 19:06, Jon circleci@discoursemail.com a écrit:


#11

Thinking about this, I wonder if there is a way to tell Node what address to start the server on? I wonder if it does the equivalent of ifconfig and binds to the first one it finds, and if so, if the LAN IP is first in the list, it binds to that rather than localhost?

Hmm, or see this note on a bug report - can you try this?

gatsby serve --host localhost

#12

@halfer that option is for gatsby develop – and yes, I already use it in that case:

{
  "scripts": {
    "dev": "gatsby develop -H localhost",
    "build": "gatsby build",
    "start": "gatsby serve",
    "format": "prettier --trailing-comma es5 --no-semi --single-quote --write 'src/**/*.js'",
    "test": "jest test/"
  }
  ...
}

But here we are talking about the actual production build.


#13

Yes, I meant serve - admittedly it was a guess. Does serve not allow you to change the host or port bindings?


#14

Gatsby relies on the serve NPM package which has no such argument. But anyway, I am able to run gatsby serve locally and access the server at http://localhost:9000, so it should also work on CircleCI.


#15

Maybe, but I am not convinced this is necessarily a CircleCI issue, for the reasons I stated earlier. No matter though, it’s fixed! I thought I would just kick around some ideas to see if it can be made to attach to localhost.

Indeed, perhaps it’s a bug in npm serve - it would be interesting to see some internal logic to see why it binds to the address it does.


#16

After glancing into the zeit/serve code, I honestly don’t know. After the server has started, it just displays that it’s listening on require('ip').address(), but don’t see where the server is configured.


#17

I was having a very similar issue. I’m using Jest/Puppeteer to do acceptance tests.

This was my exact error message in Circle Ci:

Failed to launch chrome!

My test web page is served with express. This is my specific test script related to this comment stream:

"test:acceptance": "start-server-and-test start http://localhost:3000/sandbox/ test:acceptance:cmd",

I setup puppeteer to be headless:

// browser defaults
let page
let browser
const width = 1920
const height = 1080

beforeAll(async () => {
  browser = await puppeteer.launch({
    headless: true,
    slowMo: 80,
    args: [`--window-size=${width},${height}`],
  })
  page = await browser.newPage()
  await page.setViewport({ width, height })
})
afterAll(() => {
  browser.close()
})

In my .circleci/config.yml, I set my circle image to be -browser (which is what I didn’t do initially):

defaults: &defaults
  working_directory: ~/code
  docker:
    - image: circleci/node:8-browsers
      environment:
        NPM_CONFIG_LOGLEVEL: error # make npm commands less noisy
        JOBS: max # https://gist.github.com/ralphtheninja/f7c45bdee00784b41fed

My acceptance tests are now working in CircleCi 2. Hope this comment helps or provides clarity somehow.


#18

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.