Splunk Search

How do you combine a pair of 2 fields into a column?

anu41
Explorer
I need to create a Dashboard with below columns  from below event data.   I couldn't able to get "Status" column value which is combination of  eventData{}.StatusCount{}.status and eventData{}.StatusCount{}.count
Thanks in advance!!!
 
Dashboard five columns  and  expected values:   
 
 Date : "2021-10-14",
eventKey: "event.request",
ReceivedCount: 10,
ProcessedCount: 10,
MismatchCount: 0,
Status : 
  • DOCUMENT_REQUEST_RECEIVED:10
  • DOCUMENT_SUCCESS:10
  • DOCUMENT_NOTIFY_SUCCESS:10
 
 
"eventData": [
{
"Date": "2021-10-14",
"eventKey": "event.request",
"ReceivedCount": 10,
"ProcessedCount": 10,
"MismatchCount": 0,
"StatusCount": [
{
"status": "DOCUMENT_REQUEST_RECEIVED",
"count": 10
},
{
"status": "DOCUMENT_SUCCESS",
"count": 10
},
{
"status": "DOCUMENT_NOTIFY_SUCCESS",
"count": 10
}
]
}
]
Labels (2)
0 Karma
1 Solution

yuanliu
SplunkTrust
SplunkTrust

I forgot to mvexpand the events themselves.

 

| spath path=eventData{}
| mvexpand eventData{}
| spath input=eventData{}
| eval Status = mvzip('eventStatusCount{}.status', 'eventStatusCount{}.count', ":")
| table eventDate, ReceivedCount, ProcessedCount, MismatchCount, Status
| rename ReceivedCount as "Total Event Received Count", ProcessedCount as "Total Event Processed Count", MismatchCount as "Total Event Mismatch Count"

 

  

eventDateTotal Event Received CountTotal Event Processed CountTotal Event Mismatch CountStatus
2022-10-201000100 
2022-10-2110002998
DOCUMENT_ERROR:2
DOCUMENT_REQUEST_RECEIVED:2

View solution in original post

Tags (1)
0 Karma

yuanliu
SplunkTrust
SplunkTrust

If the array eventData{}.statusCount{} is always fully populated, i.e., there is no null value for either eventData{}.StatusCount{}.status or eventData{}.StatusCount{}.count, the answer can be as simple as mvzip.

| eval Status = mvzip('eventData{}.StatusCount{}.status', 'eventData{}.StatusCount{}.count', ":")

If any of them could be null, you'll need mvexpand (as well as spath).  Something like

| spath path=eventData{}.StatusCount{}
| mvexpand eventData{}.StatusCount{}
| spath input=eventData{}.StatusCount{}
| eval Status = status . ":" count
| rename eventData{}.* as *
| stats values(Status) as Status by Date eventKey ReceivedCount ProcessedCount MismatchCount _time
``` your data illustration doesn't seem to include a unique key for eventData{} so I'm using _time ```

Hope this helps.

Tags (3)
0 Karma

anu41
Explorer

Thanks yuanliu for solution!!

The first solution mvzip function works for me, but  i didn't get values for the 'eventData{}.StatusCounr{}.status' ,  If remove the single codes (eventData{}.StatusCounr{}.status) getting  "Error in 'eval' command: The expression is malformed. Expected )."  Could you please suggest

| eval Status = mvzip('eventData{}.StatusCount{}.status', 'eventData{}.StatusCount{}.count', ":")

 

0 Karma

yuanliu
SplunkTrust
SplunkTrust

I assume that you already get eventData{}.* from Splunk?  I do notice that your sample data is missing outer brackets to make it JSON compliant.  Is that a real, complete (anonymized) event?

This is the emulation that I used:

 

| makeresults
| eval _raw="{\"eventData\": [
{
\"Date\": \"2021-10-14\",
\"eventKey\": \"event.request\",
\"ReceivedCount\": 10,
\"ProcessedCount\": 10,
\"MismatchCount\": 0,
\"StatusCount\": [
{
\"status\": \"DOCUMENT_REQUEST_RECEIVED\",
\"count\": 10
},
{
\"status\": \"DOCUMENT_SUCCESS\",
\"count\": 10
},
{
\"status\": \"DOCUMENT_NOTIFY_SUCCESS\",
\"count\": 10
}
]
}
]
}"
| spath
``` the above is data emulation ```

 

With this emulation, I get

Status
_raweventData{}.DateeventData{}.MismatchCounteventData{}.ProcessedCounteventData{}.ReceivedCount
eventData{}.StatusCount{}.count
eventData{}.StatusCount{}.status
eventData{}.eventKey
DOCUMENT_REQUEST_RECEIVED:10
DOCUMENT_SUCCESS:10
DOCUMENT_NOTIFY_SUCCESS:10
{"eventData": [ { "Date": "2021-10-14", "eventKey": "event.request", "ReceivedCount": 10, "ProcessedCount": 10, "MismatchCount": 0, "StatusCount": [ { "status": "DOCUMENT_REQUEST_RECEIVED", "count": 10 }, { "status": "DOCUMENT_SUCCESS", "count": 10 }, { "status": "DOCUMENT_NOTIFY_SUCCESS", "count": 10 } ] } ] }2021-10-1401010
10
10
10
DOCUMENT_REQUEST_RECEIVED
DOCUMENT_SUCCESS
DOCUMENT_NOTIFY_SUCCESS
event.request
0 Karma

anu41
Explorer

Hi yuanliu , Thanks again for checking!!

I am able to get data but has an issue with the "Status"  values as below and screenshot

 EventDate: 21/10/2022EventDate: 20/10/2022
Expected "Status"DOCUMENT_ERROR:2
DOCUMENT_REQUEST_RECEIVED:2
 
Actual "Status"DOCUMENT_REQUEST_RECEIVED:2DOCUMENT_ERROR:2

 

|eval Status = mvzip('eventData{}.eventStatusCount{}.status', 'eventData{}.eventStatusCount{}.count', ":") | table "eventData{}.eventDate","eventData{}.ReceivedCount",
"eventData{}.ProcessedCount","eventData{}.MismatchCount","Status"
| rename eventData{}.eventDate as "EventDate",eventData{}.ReceivedCount as "Total Event Received Count", eventData{}.ProcessedCount as "Total Event Processed Count",
eventData{}.MismatchCount as "Total Event Mismatch Count"

 

"eventData": [
{
"eventDate": "2022-10-20",
"eventKey": "event.request",
"ProcessedCount": 0,
"eventStatusCount": [],
"ReceivedCount": 100,
"MismatchCount": 100
},
{
"eventDate": "2022-10-21",
"eventKey": "event.request",
"ProcessedCount": 2,
"eventStatusCount": [
{
"status": "DOCUMENT_ERROR",
"count": 2
},
{
"status": "DOCUMENT_REQUEST_RECEIVED",
"count": 2
}
],
"ReceivedCount": 1000,
"MismatchCount": 998
}
]

anu41_1-1668510551793.png

 

0 Karma

yuanliu
SplunkTrust
SplunkTrust

I forgot to mvexpand the events themselves.

 

| spath path=eventData{}
| mvexpand eventData{}
| spath input=eventData{}
| eval Status = mvzip('eventStatusCount{}.status', 'eventStatusCount{}.count', ":")
| table eventDate, ReceivedCount, ProcessedCount, MismatchCount, Status
| rename ReceivedCount as "Total Event Received Count", ProcessedCount as "Total Event Processed Count", MismatchCount as "Total Event Mismatch Count"

 

  

eventDateTotal Event Received CountTotal Event Processed CountTotal Event Mismatch CountStatus
2022-10-201000100 
2022-10-2110002998
DOCUMENT_ERROR:2
DOCUMENT_REQUEST_RECEIVED:2
Tags (1)
0 Karma

anu41
Explorer

Thanks @yuanliu for the solution!! It is working 

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...