Test Metadata API call returning empty


#1

I am trying to get the test metadata for a build but the json response is empty.
What am I doing wrong, maybe there is not test meta data.
I am using curl https://circleci.com/api/v1/project/:username/:project/:build_num/tests?circle-token=:token
and it returns "tests" : [ ]

I am storing my test meta data from Codeception, so perhaps this is what the problem is. I am creating xml files from my codeception tests, and saving them in $CIRCLE_TEST_REPORTS/codeception/
The xml files show up in the artifacts list in the Web UI.

I have tried the the list Artifacts API call, and it works just fine, but it just shows a list of all the artifact files, not quite what I want. I can’t really use that because I need to be logged to view the files, so I’m not sure how to do that with a curl.

I suppose I need to store the test meta data some other way because it isn’t being registered properly.

Thanks
cloud


#2

Without being able to see the test results, it seems like you may be mixing artifacts and test metadata.

When you look at the build, do you see

or

Even saving the files in the test metadata folder, I suspect it won’t pull with an api call unless it’s able to parse them, so we may need to solve that first if that is the case.


#3

Yes it is the case, I see the warning message, not the junit message.
Is there a way I can get it to respond to the API call.
I am using
find . -type f -regex "wp-content/plugins/programs-tests/_output/.*xml" -exec cp {} $CIRCLE_TEST_REPORTS/codeception/ \;
to store the xml files,. I copied this from the Docs.
I noticed that it has /.*xml which seems to me like it should be /*.xml.
I will try that and see if it makes a difference.
Let me know if you have any other ideas.
Thanks drazisil


#4

The xml files are likely in the incorrect format. What program/command is generating them?

I haven’t made a wordpress plugin before.


#5

we are using Codeception, not connected to wordpress, just a PHP testing framework.
Do you know what the correct formatting is?
Here is a sample from the files I generate.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="acceptance-vagrant" tests="1" assertions="14" failures="0" errors="0" time="5.485423">
    <testcase file="/srv/www/rg/multisite/htdocs/wp-content/plugins/programs-tests///acceptance/RegPaymentCest.php" name="testAltPaymentInReg" class="RegPaymentCest" feature="register and test if alt payment is working correctly" assertions="14" time="5.485423"/>
  </testsuite>
</testsuites>

#6

I’m wondering if https://github.com/Codeception/Codeception/pull/2400 may be the issue.

Since the spec clearly states it should be classname, not class

Try this:

find . -type f -regex "wp-content/plugins/programs-tests/_output/.*xml" -exec cp {} $CIRCLE_TEST_REPORTS/codeception/ \;  find ./ -type f -exec sed -i -e 's/class=/classname=/g' {} \; 

Hope that code works, it’s completely untested and I’m not a bash programmer :smile:

Ref: https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd


#7

but you are super bad ass, that much has been established :smile:


#8

Well I tested the find and replace code, and it works, but it also generates a duplicate file with a .xml-e ending. Not sure what that is from.
I’m going to try it in CI now and see what happens.


#9

That’s what I get for copying reference commands without looking up what they do. It’s backups, I believe you drop the -i -e part if you don’t want them. :smile:


#10

weird, when I try to remove the -i -e part, or keep one or the other, no combination works. I guess having backups aren’t that bad


#11

Try -i ''

Ref: http://www.grymoire.com/Unix/Sed.html#uh-62h


#12

HUZZAH!!! It works beautifully


#13

Hey, I got it to work properly YAY. Thanks so much drazisil, you’re the greatest.
Here is what I used to get it working.
- find . -name '*.xml' -exec cp {} $CIRCLE_TEST_REPORTS/codeception/ \; - find ./ -type f -exec sed -i '' 's/class=/classname=/g' {} \;

It was the first line, it didn’t actually find anything.
I didn’t test it without your line that changes the classname, but it certainly worked with it. Got the proper message in the web ui, and I can get the results from the API call. Woot! Now I just have to make sure it does it for all the containers


#14

Fantastic! Very glad to hear. I’ve updated Test Metadata with Codeception with your updated instructions :smile:


#15