I have an XML results input that is indexed on per Test Suite. Each Test Suite has many Test Cases, and each Test Case has many Test Steps.
I am trying to create a report where we would like to find out which Test Suite, Case, and Steps failed and what was the error message. However, my search is coming back with all the test case names, and all messages regardless of failure or success. Is there a way to filter out just the failed status for a specific node?
Current search:
sourcetype=test_suite_result_xml testRunnerResults.testCase.status=FAILED | stats values(testSuiteName) values(testRunnerResults.testCase.testCaseName) values(testRunnerResults.testCase.testStepResults.result.message) by testRunnerResults.testCase.testCaseName
Sample XML data:
<testSuite>
<startTime>15:33:18</startTime>
<status>FAILED</status>
<testSuiteName>UserLifecycleManager</testSuiteName>
<timeTaken>399799</timeTaken>
<testRunnerResults>
<testCase>
<reason>Failing due to failed test step</reason>
<startTime>15:33:18</startTime>
<status>FAILED</status>
<testCaseId>f1d9066c-6744-462e-bf76-6eed9b610a5a</testCaseId>
<testCaseName>CreateUser</testCaseName>
<timeTaken>1881</timeTaken>
<testStepResults>
<result>
<message>[GetApplicationAndBaseUrl] OK: took 279 ms</message>
<name>GetApplicationAndBaseUrl</name>
<order>2</order>
<started>15:33:18.431</started>
<status>OK</status>
<timeTaken>279</timeTaken>
</result>
<result>
<message>[CreateNewUser] FAILED: took 281 ms
-> [Valid HTTP Status Codes] Response status code:400 is not in acceptable list of status codes</message>
<name>CreateNewUser</name>
<order>9</order>
<started>15:33:20.622</started>
<status>FAILED</status>
<timeTaken>281</timeTaken>
</result>
</testStepResults>
</testCase>
<testCase>
<reason></reason>
<startTime>15:33:21</startTime>
<status>FINISHED</status>
<testCaseId>f72a96f9-64f5-4ce4-861a-151e3aadd41f</testCaseId>
<testCaseName>VerifyGroup</testCaseName>
<timeTaken>598</timeTaken>
<testStepResults>
<result>
<message>[GetApplicationAndBaseUrl] OK: took 256 ms</message>
<name>GetApplicationAndBaseUrl</name>
<order>2</order>
<started>15:33:21.568</started>
<status>OK</status>
<timeTaken>256</timeTaken>
</result>
</testStepResults>
</testCase>
Have you tried the mvexpand command (http://docs.splunk.com/Documentation/Splunk/6.2.0/SearchReference/mvexpand). That may give you the results you are looking for.
..| mvexpand message
Have you setup your Search Head to accept the XML formatted data so that it extracts the fields directly? Add this to the props.conf on your Search Heads for the sourcetype set for this data:
[YOUR_SOURCETYPE]
KV_MODE = xml
Then you should be able to code your searches to look for the failed tests:
YOUR_SOURCETYPE="your_sourcetype" status="FAILED"
I'm getting a bit closer, however, still not the exact format I'l looking for...
source="testSuiteResults.xml" sourcetype="test_suite_result_xml" testSuite.status="FAILED" | xpath outfield=message "//testSuite/testRunnerResults/testCase/testStepResults/result[status="FAILED"]/message" | xpath outfield=testSuiteName "//testSuite/testSuiteName" | xpath outfield=testCaseName "//testSuite/testRunnerResults/testCase[status="FAILED"]/testCaseName" | xpath outfield=name "//testSuite/testRunnerResults/testCase/testStepResults/result[status="FAILED"]/name" | table testSuiteName, testCaseName, name, message
The result I'm getting is each Test Suite is a row with all failed Test Cases and Messages. Is it possible for each Message to be a row with the corresponding Test Case name?
Yes, and I have the breaks on the testSuite tags
[test_suite_result_xml]
DATETIME_CONFIG =
KV_MODE = xml
LINE_BREAKER = (<testSuite>)
MUST_BREAK_AFTER = \</testSuite\>
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = true
TRUNCATE = 0
pulldown_type = true
BREAK_ONLY_BEFORE = (<testSuite>)
TIME_PREFIX = <startTime>
category = Custom
disabled = false
These are (mostly) index time settings. Are your Indexers and Search Heads separate servers (distributed architecture)? If so, the KV_MODE setting of props.conf needs to be on your Search Heads.