How to make CIrcle CI 2.0 store artifacts on failed builds - TestNG

artur-abd · January 10, 2019, 7:31pm

We need it in terms of debugging failed tests on CI (create a screenshot for ex.). Right now artifacts tab in CIrcle is not visible if the build failed (because of some test)

We configured manually in each test using Selenium directly using after method:

version: 2
jobs:
  build:
    resource_class: large
    working_directory: ~/bevager/fnb
    parallelism: 1
    shell: /bin/bash -eo pipefail
    environment:
      CIRCLE_ARTIFACTS: /tmp/circleci-artifacts
      CIRCLE_TEST_REPORTS: /tmp/circleci-test-results
    # In CircleCI 1.0 we used a pre-configured image with a large number of languages and other packages.
    # In CircleCI 2.0 you can now specify your own image, or use one of our pre-configured images.
    # The following configuration line tells CircleCI to use the specified docker image as the runtime environment for you job.
    # We have selected a pre-built image that mirrors the build environment we use on
    # the 1.0 platform, but we recommend you choose an image more tailored to the needs
    # of each job. For more information on choosing an image (or alternatively using a
    # VM instead of a container) see https://circleci.com/docs/2.0/executor-types/
    # To see the list of pre-built images that CircleCI provides for most common languages see
    # https://circleci.com/docs/2.0/circleci-images/
    docker:
        - image: circleci/openjdk:8-jdk-node-browsers
        - image: mysql:5.7.21
          environment:
            - MYSQL_ROOT_PASSWORD=ubuntu
            - MYSQL_DATABASE=circle_test
    steps:    
    - checkout
    - run: mkdir -p $CIRCLE_ARTIFACTS $CIRCLE_TEST_REPORTS
    - run: git clone -b $CIRCLE_BRANCH git@github.com:bevager/fnb-tests.git
    - restore_cache:
        keys:
        # This branch if available
        - v4-dep-{{ .Branch }}-
        # Default branch if not
        - v4-dep-master-
        # Any branch if there are none on the default branch - this should be unnecessary if you have your default branch configured correctly
        - v4-dep-
    - run: sudo apt-get install libxss1 libappindicator1 libindicator7 libappindicator3-1
    - run: wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
    - run: sudo dpkg -i ./google-chrome*.deb
    - run: sudo apt-get install -f
    - run: curl https://chromedriver.storage.googleapis.com/2.43/chromedriver_linux64.zip| gzip -dc > chromedriver
    - run: chmod +x chromedriver
    - run: sudo npm install -g ember-cli
    - run: cd frontend && rm -rf tmp/
    - run: cd frontend && npm install
    - run: cd backend && ./build no-zip
    - run: sudo apt install mysql-client
    - run: mysql -h 127.0.0.1 -u root -pubuntu circle_test < test-db.sql
    - run:
        command: backend/fnbtech-LATEST-BUILD/stage/opt/docker/bin/backend -Dconfig.resource=prod.conf
        background: true
    - save_cache:
        key: v4-dep-{{ .Branch }}-{{ epoch }}
        paths:
        - vendor/bundle
        - ~/virtualenvs
        - ~/.m2
        - ~/.ivy2
        - ~/.bundle
        - ~/.gradle
        - ~/.cache/bower
        - ~/.sbt
        - ~/.ivy2
        - frontend/node_modules
        - backend/target/resolution-cache
        - backend/target/streams
        - backend/project/target/resolution-cache
        - backend/project/target/streams
    - run: if (git log --format=%B -n 1 $CIRCLE_SHA1) | grep -iqF release; then cd fnb-tests/FNB-Automation/FNB_Testing && mvn clean && mvn test -e -X -Dwebdriver.chrome.driver=../../../chromedriver -Dsurefire.suiteXmlFiles=regression.xml; elif (git log --format=%B -n 1 $CIRCLE_SHA1) | grep -iqF debug; then cd fnb-tests/FNB-Automation/FNB_Testing && mvn clean && mvn test -e -X -Dwebdriver.chrome.driver=../../../chromedriver -Dsurefire.suiteXmlFiles=debug.xml; else cd fnb-tests/FNB-Automation/FNB_Testing && mvn clean && mvn test -e -X -Dwebdriver.chrome.driver=../../../chromedriver -Dsurefire.suiteXmlFiles=smoke.xml; fi
    - run: mkdir -p $CIRCLE_TEST_REPORTS/junit/
    - run: find . -type f -regex ".*/target/surefire-reports/.*xml" -exec cp {} $CIRCLE_TEST_REPORTS/junit/ \;
     - store_test_results:
        path: /tmp/circleci-test-results
    - store_artifacts:
        path: /tmp/circleci-artifacts
    - store_artifacts:
        path: target/universal
    - store_artifacts:
        path: /tmp/circleci-test-results

Maven file:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.FNB</groupId>
	<artifactId>FNB_Testing</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<packaging>jar</packaging>

	<name>FNB_Testing</name>
	<url>http://maven.apache.org</url>

	<properties>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
			
	</properties>

	<dependencies>

		<dependency>
			<groupId>org.seleniumhq.selenium</groupId>
			<artifactId>selenium-java</artifactId>
			<version>3.4.0</version>
		</dependency>

		<dependency>
			<groupId>org.testng</groupId>
			<artifactId>testng</artifactId>
			<version>6.11</version>
			<scope>test</scope>
		</dependency>
		<!-- <dependency>
			<groupId>com.sun.mail</groupId>
			<artifactId>javax.mail</artifactId>
			<version>1.6.0</version>
		</dependency>
		<dependency>
			<groupId>javax.mail</groupId>
			<artifactId>javax.mail-api</artifactId>
			<version>1.6.0</version>
		</dependency>

		<dependency>
			<groupId>com.relevantcodes</groupId>
			<artifactId>extentreports</artifactId>
			<version>2.41.2</version>
		</dependency> -->


	</dependencies>



	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>2.22.1</version>

				<configuration>
					<forkCount>0</forkCount>
					<argLine>-Xms1024m -Xmx2048m</argLine>

					<suiteXmlFiles>
						<suiteXmlFile>regression.xml</suiteXmlFile>
						<suiteXmlFile>smoke.xml</suiteXmlFile>
					</suiteXmlFiles>
				</configuration>
			</plugin>
		</plugins>
</build>

</project>

halfer · January 10, 2019, 8:46pm

It should work, yes. You can make image files available using “test/build artifacts” - see the documentation.

artur-abd · January 25, 2019, 11:59pm

I didn’t find any information related to how to configure it using Java + TestNG

halfer · January 26, 2019, 1:34am

Right. You could ask a question on Stack Overflow, though it would have to be a great deal more detailed than this - it is too broad and brief to answer.

Certainly, you can ask questions here too, but the audience is much smaller. Either way, readers will want you to have tried something beforehand. I would expect your desired test stack can be gotten to work on CircleCI.

artur-abd · January 28, 2019, 6:38pm

Created on StackOverflow and Updated topic with more information
Thank you Jon

halfer · January 28, 2019, 7:24pm

No worries.

Topic created here.

halfer · January 28, 2019, 7:43pm

Reading your Stack Overflow question, I wonder if it will be a bit confusing for readers. CircleCI has no capacity to take screenshots at all, so you are asking for something that cannot be done.

However, test software has this capacity. Selenium can do it, PhantomJS can do it, I’m sure many others can do it. That’s why I suggested it is more suitable for a broad audience - I didn’t think it was particularly a CircleCI question.

How you ask your questions is up to you, but I would be inclined to strip CircleCI (and config) out of that question and ask how to take a screenshot on your local machine. Have you tried that yet, as a first milestone?

artur-abd · February 27, 2019, 1:29am

Thank you Halfer. Yes, it’s very easy.

In after method:
`

public void afterMethod(ITestResult testResult) throws IOException, InterruptedException {
        if (testResult.getStatus() == ITestResult.FAILURE) {
            Screenshots.takeScreenshot(driver, testResult.getName());
            System.out.println("======= Test Failed =======");
            File src= ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
            try {
                FileUtils.copyFile(src, new File("test_edi_invoices/" + this.getClass().getName()));
            } catch (IOException e) {
                System.out.println(e.getMessage());
            }
        }
    }

artur-abd · March 1, 2019, 12:17am

Hi Halfer,
updated title and info in first message

artur-abd · March 1, 2019, 1:26am

pom.xml:

4.0.0

<groupId>com.FNB</groupId>
<artifactId>FNB_Testing</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>

<name>FNB_Testing</name>
<url>http://maven.apache.org</url>

<properties>
	<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		
</properties>

<dependencies>

	<dependency>
		<groupId>org.seleniumhq.selenium</groupId>
		<artifactId>selenium-java</artifactId>
		<version>3.4.0</version>
	</dependency>

	<dependency>
		<groupId>org.testng</groupId>
		<artifactId>testng</artifactId>
		<version>6.11</version>
		<scope>test</scope>
	</dependency>
	<!-- <dependency>
		<groupId>com.sun.mail</groupId>
		<artifactId>javax.mail</artifactId>
		<version>1.6.0</version>
	</dependency>
	<dependency>
		<groupId>javax.mail</groupId>
		<artifactId>javax.mail-api</artifactId>
		<version>1.6.0</version>
	</dependency>

	<dependency>
		<groupId>com.relevantcodes</groupId>
		<artifactId>extentreports</artifactId>
		<version>2.41.2</version>
	</dependency> -->


</dependencies>



<build>
	<plugins>
		<plugin>
			<groupId>org.apache.maven.plugins</groupId>
			<artifactId>maven-surefire-plugin</artifactId>
			<version>2.22.1</version>

			<configuration>
				<forkCount>0</forkCount>
				<argLine>-Xms1024m -Xmx2048m</argLine>

				<suiteXmlFiles>
					<suiteXmlFile>regression.xml</suiteXmlFile>
					<suiteXmlFile>smoke.xml</suiteXmlFile>
				</suiteXmlFiles>
			</configuration>
		</plugin>
	</plugins>

halfer · March 2, 2019, 1:52pm

It is not clear what this post is asking. It was originally posted to ask about whether CircleCI can do screenshots. It seems now to have been modified to ask a different question, rendering my answers redundant. Then new XML code snippets have been added without any explanation as to what they are for.

I recommend that you roll back your first post to the original question, and then start a new thread for your new topic. Remember that once you solve a problem on the web, there is an expectation that you leave that help available for future readers - do not destroy it because you have received the help you needed.

artur-abd · March 21, 2019, 10:27pm

finally changed location in after method of each test and it started working

Topic		Replies	Views
How do I configure Circle to accept pending tests when evaluating a build? Build Environment rails , ruby , rspec	3	3139	August 2, 2019
Can't see test summary Running Tests nodejs , circle-yml	1	1385	June 18, 2018
Jest JUnit store_test_results Build Environment circle-yml	5	6611	June 3, 2019
Tests randomly fail on CircleCI 2.0 Build Environment	8	3114	July 31, 2018
How do I configure test reporting with Cypress? Running Tests integration-tests	3	8156	August 6, 2018

How to make CIrcle CI 2.0 store artifacts on failed builds - TestNG

Related Topics