Data search in JSON format

[arttifex]

Good day. I did not find the answer to my question, so I made a new topic. My device sends data from IDS in JSON format. I get this data in Splunk. The data is presented below in the screenshot. I try to search on various data, but it doesn’t work. I see that the search is successful, but I don’t see the fields I need. In the "questions and answers" section, I found a recommendation to change the value of the "KV_MODE" field, but this did not help. Please tell me what I'm doing wrong. What I need to pay attention to.

The data has the following structure:

Apr 22 18:09:26 172.20.8.2 Apr 22 18:09:54 vpnfw dmutmd[1505]:
{

"source": "IDS",

"message":
{
"alert":
{
"action":"allowed",
"category":"A Network Trojan was detected",
"gid":1,
"metadata":
{

"affected_product":["Windows_XP_Vista_7_8_10_Server_32_64_Bit"],
"attack_target":["Client_Endpoint"],
"created_at":["2015_03_13"],
"deployment":["Perimeter"],
"former_category":["ADWARE_PUP"],
"group_tss":["2"],
"malware_family":["Loadmoney"],
"performance_impact":["Low"],
"priority":["1"],
"signature_severity":["Minor"],
"tag":["Loadmoney"],
"updated_at":["2019_10_07"]
},
"rev":8,
"severity":1,
"signature":"ET MALWARE Loadmoney User Agent",
"signature_id":2024249
},
"app_proto":"http",
"dest_ip":"5.9.80.173",
"dest_port":80,
"event_type":"alert"
}
}

I'm doing a search with the following query: "IDS | spath path=message.alert.category" and get a result.

[DalJeanis]

The long term solution is to fix your ingestion and props so that JSONs are automatically extracted. In the meantime, you can use this code sample for finding your immediate solutions. A full run-anywhere example is after the break.

| rename COMMENT as "the below reformats that data so you can look at it."
| rex "^[\s\S]*?(?<myJSON>{[\s\S]*})"
| spath input=myJSON

---

Here's the full run-anywhere sample for anyone who wants to play with it.

|makeresults 
| rename COMMENT as "The above creates one dummy record, and the below inputs your sample data"
| eval _raw  = "Apr 22 18:09:26 172.20.8.2 Apr 22 18:09:54 vpnfw dmutmd[1505]:
{
\"source\": \"IDS\",
\"message\":
{
\"alert\":
{
\"action\":\"allowed\",
\"category\":\"A Network Trojan was detected\",
\"gid\":1,
\"metadata\":
{
\"affected_product\":[\"Windows_XP_Vista_7_8_10_Server_32_64_Bit\"],
\"attack_target\":[\"Client_Endpoint\"],
\"created_at\":[\"2015_03_13\"],
\"deployment\":[\"Perimeter\"],
\"former_category\":[\"ADWARE_PUP\"],
\"group_tss\":[\"2\"],
\"malware_family\":[\"Loadmoney\"],
\"performance_impact\":[\"Low\"],
\"priority\":[\"1\"],
\"signature_severity\":[\"Minor\"],
\"tag\":[\"Loadmoney\"],
\"updated_at\":[\"2019_10_07\"]
},
\"rev\":8,
\"severity\":1,
\"signature\":\"ET MALWARE Loadmoney User Agent\",
\"signature_id\":2024249
},
\"app_proto\":\"http\",
\"dest_ip\":\"5.9.80.173\",
\"dest_port\":80,
\"event_type\":\"alert\"
}
}"
| rename COMMENT as "The above just inputs your sample data"

| rename COMMENT as "the below reformats that data so you can look at it."
| rex "^[\s\S]*?(?<myJSON>{[\s\S]*})"
| spath input=myJSON

[to4kawa]

see my Q&A
https://answers.splunk.com/answers/802709/make-extractions-in-propsconf-from-search-query.html