Splunk Search

How do you combine a pair of 2 fields into a column?

anu41
Explorer
I need to create a Dashboard with below columns  from below event data.   I couldn't able to get "Status" column value which is combination of  eventData{}.StatusCount{}.status and eventData{}.StatusCount{}.count
Thanks in advance!!!
 
Dashboard five columns  and  expected values:   
 
 Date : "2021-10-14",
eventKey: "event.request",
ReceivedCount: 10,
ProcessedCount: 10,
MismatchCount: 0,
Status : 
  • DOCUMENT_REQUEST_RECEIVED:10
  • DOCUMENT_SUCCESS:10
  • DOCUMENT_NOTIFY_SUCCESS:10
 
 
"eventData": [
{
"Date": "2021-10-14",
"eventKey": "event.request",
"ReceivedCount": 10,
"ProcessedCount": 10,
"MismatchCount": 0,
"StatusCount": [
{
"status": "DOCUMENT_REQUEST_RECEIVED",
"count": 10
},
{
"status": "DOCUMENT_SUCCESS",
"count": 10
},
{
"status": "DOCUMENT_NOTIFY_SUCCESS",
"count": 10
}
]
}
]
Labels (2)
0 Karma
1 Solution

yuanliu
SplunkTrust
SplunkTrust

I forgot to mvexpand the events themselves.

 

| spath path=eventData{}
| mvexpand eventData{}
| spath input=eventData{}
| eval Status = mvzip('eventStatusCount{}.status', 'eventStatusCount{}.count', ":")
| table eventDate, ReceivedCount, ProcessedCount, MismatchCount, Status
| rename ReceivedCount as "Total Event Received Count", ProcessedCount as "Total Event Processed Count", MismatchCount as "Total Event Mismatch Count"

 

  

eventDateTotal Event Received CountTotal Event Processed CountTotal Event Mismatch CountStatus
2022-10-201000100 
2022-10-2110002998
DOCUMENT_ERROR:2
DOCUMENT_REQUEST_RECEIVED:2

View solution in original post

Tags (1)
0 Karma

yuanliu
SplunkTrust
SplunkTrust

If the array eventData{}.statusCount{} is always fully populated, i.e., there is no null value for either eventData{}.StatusCount{}.status or eventData{}.StatusCount{}.count, the answer can be as simple as mvzip.

| eval Status = mvzip('eventData{}.StatusCount{}.status', 'eventData{}.StatusCount{}.count', ":")

If any of them could be null, you'll need mvexpand (as well as spath).  Something like

| spath path=eventData{}.StatusCount{}
| mvexpand eventData{}.StatusCount{}
| spath input=eventData{}.StatusCount{}
| eval Status = status . ":" count
| rename eventData{}.* as *
| stats values(Status) as Status by Date eventKey ReceivedCount ProcessedCount MismatchCount _time
``` your data illustration doesn't seem to include a unique key for eventData{} so I'm using _time ```

Hope this helps.

Tags (3)
0 Karma

anu41
Explorer

Thanks yuanliu for solution!!

The first solution mvzip function works for me, but  i didn't get values for the 'eventData{}.StatusCounr{}.status' ,  If remove the single codes (eventData{}.StatusCounr{}.status) getting  "Error in 'eval' command: The expression is malformed. Expected )."  Could you please suggest

| eval Status = mvzip('eventData{}.StatusCount{}.status', 'eventData{}.StatusCount{}.count', ":")

 

0 Karma

yuanliu
SplunkTrust
SplunkTrust

I assume that you already get eventData{}.* from Splunk?  I do notice that your sample data is missing outer brackets to make it JSON compliant.  Is that a real, complete (anonymized) event?

This is the emulation that I used:

 

| makeresults
| eval _raw="{\"eventData\": [
{
\"Date\": \"2021-10-14\",
\"eventKey\": \"event.request\",
\"ReceivedCount\": 10,
\"ProcessedCount\": 10,
\"MismatchCount\": 0,
\"StatusCount\": [
{
\"status\": \"DOCUMENT_REQUEST_RECEIVED\",
\"count\": 10
},
{
\"status\": \"DOCUMENT_SUCCESS\",
\"count\": 10
},
{
\"status\": \"DOCUMENT_NOTIFY_SUCCESS\",
\"count\": 10
}
]
}
]
}"
| spath
``` the above is data emulation ```

 

With this emulation, I get

Status
_raweventData{}.DateeventData{}.MismatchCounteventData{}.ProcessedCounteventData{}.ReceivedCount
eventData{}.StatusCount{}.count
eventData{}.StatusCount{}.status
eventData{}.eventKey
DOCUMENT_REQUEST_RECEIVED:10
DOCUMENT_SUCCESS:10
DOCUMENT_NOTIFY_SUCCESS:10
{"eventData": [ { "Date": "2021-10-14", "eventKey": "event.request", "ReceivedCount": 10, "ProcessedCount": 10, "MismatchCount": 0, "StatusCount": [ { "status": "DOCUMENT_REQUEST_RECEIVED", "count": 10 }, { "status": "DOCUMENT_SUCCESS", "count": 10 }, { "status": "DOCUMENT_NOTIFY_SUCCESS", "count": 10 } ] } ] }2021-10-1401010
10
10
10
DOCUMENT_REQUEST_RECEIVED
DOCUMENT_SUCCESS
DOCUMENT_NOTIFY_SUCCESS
event.request
0 Karma

anu41
Explorer

Hi yuanliu , Thanks again for checking!!

I am able to get data but has an issue with the "Status"  values as below and screenshot

 EventDate: 21/10/2022EventDate: 20/10/2022
Expected "Status"DOCUMENT_ERROR:2
DOCUMENT_REQUEST_RECEIVED:2
 
Actual "Status"DOCUMENT_REQUEST_RECEIVED:2DOCUMENT_ERROR:2

 

|eval Status = mvzip('eventData{}.eventStatusCount{}.status', 'eventData{}.eventStatusCount{}.count', ":") | table "eventData{}.eventDate","eventData{}.ReceivedCount",
"eventData{}.ProcessedCount","eventData{}.MismatchCount","Status"
| rename eventData{}.eventDate as "EventDate",eventData{}.ReceivedCount as "Total Event Received Count", eventData{}.ProcessedCount as "Total Event Processed Count",
eventData{}.MismatchCount as "Total Event Mismatch Count"

 

"eventData": [
{
"eventDate": "2022-10-20",
"eventKey": "event.request",
"ProcessedCount": 0,
"eventStatusCount": [],
"ReceivedCount": 100,
"MismatchCount": 100
},
{
"eventDate": "2022-10-21",
"eventKey": "event.request",
"ProcessedCount": 2,
"eventStatusCount": [
{
"status": "DOCUMENT_ERROR",
"count": 2
},
{
"status": "DOCUMENT_REQUEST_RECEIVED",
"count": 2
}
],
"ReceivedCount": 1000,
"MismatchCount": 998
}
]

anu41_1-1668510551793.png

 

0 Karma

yuanliu
SplunkTrust
SplunkTrust

I forgot to mvexpand the events themselves.

 

| spath path=eventData{}
| mvexpand eventData{}
| spath input=eventData{}
| eval Status = mvzip('eventStatusCount{}.status', 'eventStatusCount{}.count', ":")
| table eventDate, ReceivedCount, ProcessedCount, MismatchCount, Status
| rename ReceivedCount as "Total Event Received Count", ProcessedCount as "Total Event Processed Count", MismatchCount as "Total Event Mismatch Count"

 

  

eventDateTotal Event Received CountTotal Event Processed CountTotal Event Mismatch CountStatus
2022-10-201000100 
2022-10-2110002998
DOCUMENT_ERROR:2
DOCUMENT_REQUEST_RECEIVED:2
Tags (1)
0 Karma

anu41
Explorer

Thanks @yuanliu for the solution!! It is working 

0 Karma
Get Updates on the Splunk Community!

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

🔐 Trust at Every Hop: How mTLS in Splunk Enterprise 10.0 Makes Security Simpler

From Idea to Implementation: Why Splunk Built mTLS into Splunk Enterprise 10.0  mTLS wasn’t just a checkbox ...