topic Re: Search numbers of messages received by application based on json message in Splunk Search

Search numbers of messages received by application based on json message

Punnu — Fri, 14 Mar 2025 03:44:07 GMT

Hello All,

This is my first post . I have just started learning writing splunk query .

Ok so we have one application sitting in kubernates cluster . We are calling end point of application and doing some activity . I am seeing in logs json which we sent while calling endpoint.

{
"header": {
"version": "1.0",
"sender": "ABC",
"publishDateTime": "2025-03-12T15:54:32Z"
},
"audit": {
"addDateTime": "2024-04-19 05:42:57",
"addBy": "PP"
}

}

I want to find count of all request I have made where I am seeing messages as addBy as PP

I was trying to use multiple things like spath search but not getting how to do .

kubernetes_cluster="abc*" index="aaaa" sourcetype = "kubernetes_logs" source = *pub-sub* |spath output=myfiled path=audit.addBy | stats count by myfiled

Re: Search numbers of messages received by application based on json message

bowesmana — Fri, 14 Mar 2025 04:17:52 GMT

Based on your example data, that would appear to work.

If you copy in this example search you can see your spath and stats command do indeed extract the correct data and give you a count of 1, so what is your problem? Are you saying this is not working for you?

If not, it would indicate your data is perhaps not as you have shown.

Re: Search numbers of messages received by application based on json message

yuanliu — Fri, 14 Mar 2025 06:12:56 GMT

Like @bowesmana says, if your data is as you illustrated, and if your search is exactly like you have shown, the search should give you the correct results. So, my speculation is that in your real search, spelling of myfiled in spath and in stats are different. For example, maybe your actual search was spelled like

|spath output=myfiled path=audit.addBy | stats count by myfield

By the way, there should be no need for spath as @bowesmana says. This search should give you exactly the same result

kubernetes_cluster="abc*" index="aaaa" sourcetype = "kubernetes_logs" source = *pub-sub* | stats count by audit.addBy

Re: Search numbers of messages received by application based on json message

Punnu — Fri, 14 Mar 2025 15:34:30 GMT

Hi All,

Due to security I was not sharing complete message . So below mentioned event is getting returned when I run following query :

kubernetes_cluster="aa*" index="aa" sourcetype = "kubernetes_logs" source = *aa* | where (WebserviceAudit="abc" ) and (caller_ip ="def" )

Now I want to further filter the data by saying where audit.addby =pp and then count of such events

Let me share snapshot

Re: Search numbers of messages received by application based on json message

yuanliu — Fri, 14 Mar 2025 19:22:47 GMT

Due to security I was not sharing complete message . So below mentioned event is getting returned

You can post sample event in text after sanitation, that is, replacing sensitive information, be it field name or field value, with fake strings. The key is to preserve structure of data, such as punctuation and other "major separators".

Based on your screenshot, it is clear that your events themselves are not in JSON. That is why the field audit.addBy is not present at search time. This is also why your spath command has no effect. On the other hand, your events do contain a JSON message.

What you need is to first extract that JSON message. Try this.

kubernetes_cluster="abc*" index="aaaa" sourcetype = "kubernetes_logs" source = *pub-sub* | rex "^[^\{]+(?<json_portion>\{.+\})" | spath input=json_portion path=audit.addBy | stats count by audit.addBy