Build goes green even though my testsuite fails (Create React App)

Build Environment Programming Languages
,
1

I’m testing my React application made with Create React App. My test command (npm test) executes react-scripts test.

No matter the result of my tests the step will succeed and never fail no matter if one or all of the tests fails.

Build errors are reported correctly and will fail the build if they happens but exit codes from react-scripts seems to be catched somehow.

Excuting the react-scripts command directly (node_modules/react-scripts/bin/react-scripts.js test --env=jsdom) doesn’t change anything.

As attempt in debugging 'I’ve tried to npm test && echo $? to see the exit code and it is always 0.
I can not reproduce this locally so I wonder if Circle is wrapping the build step somehow and that way captures the exitcode from npm test.

Screenshot 2017-09-20 10.50.59.png3062×1278 272 KB

1 Like
2

I solved this myself by explicitly installing jest.

As I’m using Create React App the jest package is an implicit dependency of my project and I wouldn’t expect it to be necessary to install it explicitly. But installing it manually in my circle config seems to fix the problem. Why the problem exists at first is still a mystery to me.

3

@hrmoller - can you give a little more detail on exactly how you did that? I added a build step (npm install jest), and it didn’t make a difference. Circle still passes builds even though jest is failing.

4

@mbifulco Actually I just experienced my solution mentioned above regressed so now I changed my test command in config.yml to this: command: npm run test -- --bail --ci which seems to work.
Actually I’m running my tests twice:

  1. npm run test -- --ci --coverage to get a full coverage report and a full report of failing tests in the Circle UI
  2. npm run test -- --bail --ci to force my build to fail if any test fails.
5

Got it, thanks @hrmoller . I gave that a shot, but still no dice. Is your test script running react-scripts test --env=jsdom, or do you have it set up to run an explicit jest command?

6

Yup this is my test command in package.json: "test": "react-scripts test --env=jsdom",

And just to get the context, this is my steps related to testing in my config.yml:

- run:
          name: Install test tools
          command: npm install jest-junit jest-cli jest

      # run coverage -- this is done without failing the build if any errors occurs to ensure a testreport is created
      - run:
          name: Coverage
          command: mkdir /tmp/reports && JEST_JUNIT_OUTPUT="/tmp/reports/test-report.xml" npm run test -- --ci --coverage --testResultsProcessor="jest-junit"

      - store_test_results:
          path: /tmp/reports

      # run tests any errors will fail the build
      - run:
          name: Tests - Jest
          command: npm run test -- --bail --ci

edit: Using react-scripts version 1.0.14

7

npm run test -- --bail --ci . It’s not working for me.

I have e completely same issue:

8

@Rutulpatel7077: as per your last post, that is not nearly enough information for any reader to be able to help you. When asking questions on the web, be detailed and expansive, and show what efforts you have got to in order to resolve the problem.

This article is a good explanation of the problems with the phrase “it’s not working”.

9

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.