Splunk Search

Jenkins JSON test results into table by build

JykkeDaMan
Path Finder

I'm wondering if the following table structure is possible (without custom JS).

Raw events are from Jenkins plugin. Below one event example. I get one per job/test/build.

 

 

{ [-]
   build_number: 615
   build_url: job1
   event_tag: build_report
   job_name: jobname
   job_result: FAILURE
   metadata: { [+]
   }
   page_num: 1
   testsuite: { [-]
     duration: 1968
     errors: 0
     failures: 6
     passes: 7
     skips: 0
     testcase: [ [-]
       { [-]
         classname: Testsuites.MyCallCenter
         duration: 122
         failedsince: 0
         groupname: Testsuites
         skipped: false
         status: PASSED
         testname: Login
         uniquename: Testsuites.MyCallCenter.Login
       }
       { [-]
         classname: Testsuites.MyCallCenter
         duration: 148
         failedsince: 0
         groupname: Testsuites
         skipped: false
         status: PASSED
         testname: Edit Calendar
         uniquename: Testsuites.MyCallCenter.Edit Calendar
       }
       { [+]
       }
     ]
     tests: 13
     time: 1968
     total: 13
   }
   user: (timer)
}

 

 

 

Can I create a table, which has a dynamic build number in the header somehow like this?

 

 

classname | testname | build-613 | build-614 | build-615
Testsuites.MyCallCenter | Login | PASSED | PASSED | PASSED
Testsuites.MyCallCenter | Edit Calendar | PASSED | FAILED | PASSED

 

 

 

Labels (1)
0 Karma
1 Solution

to4kawa
Ultra Champion
| rename COMMENT as "the logic"
| streamstats count as session
| spath build_number
| spath testsuite.testcase{} output=testcase
| stats values(eval("build_number_".build_number)) as build_number by session testcase
| spath input=testcase
| table build_number classname testname status
| eval tmp=classname."_".testname
| xyseries tmp build_number status
| eval classname=mvindex(split(tmp,"_"),0), testname=mvindex(split(tmp,"_"),1)
| fields - tmp 
| table classname testname *

Have you tried my logic? the result is wrong.  "build_number_" is missing.

If it is changed in a strange way, it cannot be fixed.

 

View solution in original post

0 Karma

JykkeDaMan
Path Finder

One thing: If I have build numbers like 4x, 5x, 3xx, the xyseries seems to sort them as string, not as a number.

Now I get: 3xx, 4x, 5x
I would like to have: 4x, 5x, 3xx, 4xx, 5xx

Screenshot 2020-10-25 at 23.45.04.jpg

0 Karma

to4kawa
Ultra Champion

sample:

| makeresults count=20
| eval name=random() % 200
| eval fieldA="TEST", fieldB="TEST2"
| xyseries fieldA name fieldB
| transpose 0 header_field=fieldA
| sort 0  column
| transpose 0 header_field=column

try transpose

0 Karma

to4kawa
Ultra Champion

 

index=_internal |head 1 | fields _raw |eval _raw="{\"build_number\":\"615\",\"build_url\":\"job1\",\"event_tag\":\"build_report\",\"job_name\":\"jobname\",\"job_result\":\"FAILURE\",\"metadata\":{},\"page_num\":\"1\",\"testsuite\":{\"duration\":\"1968\",\"errors\":\"0\",\"failures\":\"6\",\"passes\":\"7\",\"skips\":\"0\",\"testcase\":[{\"classname\":\"Testsuites.MyCallCenter\",\"duration\":\"122\",\"failedsince\":\"0\",\"groupname\":\"Testsuites\",\"skipped\":\"false\",\"status\":\"PASSED\",\"testname\":\"Login\",\"uniquename\":\"Testsuites.MyCallCenter.Login\"},{\"classname\":\"Testsuites.MyCallCenter\",\"duration\":\"148\",\"failedsince\":\"0\",\"groupname\":\"Testsuites\",\"skipped\":\"false\",\"status\":\"PASSED\",\"testname\":\"Edit Calendar\",\"uniquename\":\"Testsuites.MyCallCenter.Edit Calendar\"},{}],\"tests\":\"13\",\"time\":\"1968\",\"total\":\"13\"},\"user\":\"(timer)\"}"
| rename COMMENT as "the logic"
| spath build_number
| spath testsuite.testcase{} output=testcase
| stats values(eval("build_number_".build_number)) as build_number by testcase
| spath input=testcase
| table build_number classname testname status
| eval tmp=classname."_".testname
| xyseries tmp build_number status
| eval classname=mvindex(split(tmp,"_"),0), testname=mvindex(split(tmp,"_"),1)
| fields - tmp
| table classname testname *

I would like you to present _raw. I have to process it.

 

0 Karma

JykkeDaMan
Path Finder

Screenshot 2020-10-24 at 15.43.36.jpg

 

 

Close already. I put the build number only to short the output.

0 Karma

JykkeDaMan
Path Finder
{"event_tag":"build_report","metadata":{"scm":"git"},"job_name":"job1","testsuite":{"failures":0,"passes":2,"skips":0,"total":2,"duration":1968.0,"tests":2,"time":1968.0,"errors":0,"testcase":[{"duration":122.0,"classname":"Testsuites.MyCallCenter","testname":"Login","groupname":"Testsuites","skipped":false,"failedsince":0,"uniquename":"Testsuites.MyCallCenter.myCallCenter basic tests.Login","status":"PASSED"},
{"duration":148.0,"classname":"Testsuites.MyCallCenter","testname":"Edit Calendar","groupname":"Testsuites","skipped":false,"failedsince":0,"uniquename":"Testsuites.MyCallCenter.myCallCenter basic tests.Edit Calendar","status":"PASSED"}]
},"build_number":615,"page_num":1,"job_result":"PASSED","user":"(timer)","build_url":"job/build/615/"}
0 Karma

to4kawa
Ultra Champion
| rename COMMENT as "the logic"
| streamstats count as session
| spath build_number
| spath testsuite.testcase{} output=testcase
| stats values(eval("build_number_".build_number)) as build_number by session testcase
| spath input=testcase
| table build_number classname testname status
| eval tmp=classname."_".testname
| xyseries tmp build_number status
| eval classname=mvindex(split(tmp,"_"),0), testname=mvindex(split(tmp,"_"),1)
| fields - tmp 
| table classname testname *

Have you tried my logic? the result is wrong.  "build_number_" is missing.

If it is changed in a strange way, it cannot be fixed.

 

0 Karma

JykkeDaMan
Path Finder

Yes, the first example dit not work, I only removed the prefix "build_number_" to make the output shorter.

But your second example looks better with the split by 'session'. Now we get one row per build there. I'll look into and get back to you. 

0 Karma

JykkeDaMan
Path Finder

I think  I have some testcases larger than 5000 characters and spath only parses the first 5K?
I have 6K of chars, in case of error happens.

BTW, does it help, if I already have index time extractions defined for sourcetype?

[json:jenkins]
pulldown_type = true
INDEXED_EXTRACTIONS = json
KV_MODE = none
TRUNCATE = 0
category = Structured
LEARN_MODEL = false
description = JavaScript Object Notation format.

So my events have already to following fields:

job_name
job_result
level
linecount
log_source
log_thrown
message
metadata.scm
page_num
testsuite.duration
testsuite.errors
testsuite.failures
testsuite.passes
testsuite.skips
testsuite.testcase{}.classname
testsuite.testcase{}.duration
testsuite.testcase{}.errordetails
testsuite.testcase{}.errorstacktrace
testsuite.testcase{}.failedsince
testsuite.testcase{}.groupname
testsuite.testcase{}.skipped
testsuite.testcase{}.status
testsuite.testcase{}.testname
testsuite.testcase{}.uniquename
testsuite.tests
testsuite.time
testsuite.total
0 Karma

to4kawa
Ultra Champion
...
| streamstats count as session
| stats list(testsuite.testcase{}.*) as * by session build_number
| eval tmp=mvzip(classname,mvzip(testname,status))
| stats values(build_number) as build_number by session tmp
| eval status=mvindex(split(tmp,","),-1)
| xyseries tmp build_number status
| rex mode=sed field=tmp "s/,\w+$//g"
| rename COMMENT as "check this result"
| selfjoin tmp
| eval classname=mvindex(split(tmp,","),0), testname=mvindex(split(tmp,","),1)
| fields - tmp
| table *name *

It's very hard to create a log when there's only one log on display, isn't it?


0 Karma

JykkeDaMan
Path Finder

This is what I have now and it is looking good. I'm still playing and verifying it.

 

...
| streamstats count as session
| spath output=test "testsuite.testcase{}" 
| fields session test host job_name build_number 
| mvexpand test 
| eval result=if(isnotnull(errordetails), "Failed", if(skipped=="true", "Skipped", "Passed"))
| eval duration=round(duration, 2)
| eval result=result
| stats values(build_number) as build_number by session test
| spath input=test output=status "status" 
| spath input=test output=name "name" 
| spath input=test output=testname "testname" 
| eval testname=coalesce(name, testname) 
| spath input=test output=classname "classname" 
| spath input=test output=errordetails "errordetails" 
| spath input=test output=skipped "skipped" 
| eval result=if(isnotnull(errordetails), "Failed", if(skipped=="true", "Skipped", "Passed"))
| table build_number classname testname result
| eval tmp=classname."_".testname
| xyseries tmp build_number result
| eval classname=mvindex(split(tmp,"_"),0), testname=mvindex(split(tmp,"_"),1)
| fields - tmp 
| table classname testname *

 

Screenshot 2020-10-25 at 10.14.30.jpg

 

And your latest example look good as well :-).

Screenshot 2020-10-25 at 10.12.12.jpg

 I will accept your answer after I have verified a bit.

Thanks A LOT!

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...