Splunk Search

How to extract fields from my string logs

kotig
Path Finder

I have something like below logged in as a message.
How can i replace "This is my logfile ** ->" with empty and then how to extract name, startdate, dept, enddate, status, id and get the values.
This is my logfile ** ->
myfulljson {
name {
value: "Test"
}
startdate {
value: "2020-02-21"
}
dept {
value: 110
}
enddate {
value: "20200220"
}
status {
value: "finish"
}
id {
value: "1234"
}
}

0 Karma

woodcock
Esteemed Legend

Your trash log is not even CLOSE to valid JSON.
I used this tool to figure out how to fix it:
https://jsonlint.com/
Then I ended up with this:

| makeresults 
| eval _raw="This is my logfile ->
 myfulljson {
 name {
 value: \"Test\"
     }
 startdate {
 value: \"2020-02-21\"
     }
 dept {
 value: 110
 }
 enddate {
 value: \"20200220\"
     }
 status {
 value: \"finish\"
     }
 id {
 value: \"1234\"
     }
 }"
| rename COMMENT AS "Strip off the header garbage"
| rex mode=sed "s/^This is my logfile\s*\->[\r\n\s]*myfulljson[\r\n\s]*//"
| rename COMMENT AS "Transform the non-JSON to mostly-compliant-JSON"
| rex mode=sed "s/(\w+)([\r\n\s]*\{)/\"\1\":\2/g s/value: /\"value\": /g s/\}([\r\n\s]*\")/},\1/g"
| rename COMMENT AS "OKish JSON is now parseable with 'KV_MODE=json'"
| kv

woodcock
Esteemed Legend

To do this properly, you should transform this garbage into fully-compliant JSON (that means removing ALL whitesapce) BEFORE it is indexed. This will save you license and allow KV_MODE=json to "just work".

0 Karma

to4kawa
Ultra Champion

UPDATE:

| makeresults 
| eval _raw="This is my logfile ->
myfulljson {
name {
value: \"Test\"
}
startdate {
value: \"2020-02-21\"
}
dept {
value: 110
}
enddate {
value: \"20200220\"
}
status {
value: \"finish\"
}
id {
value: \"1234\"
}
}" 
| rex max_match=0 "(?m)(?<fieldname>[a-z]+)\s{\s*value:\s(?<fieldvalue>\S+)" 
| eval fieldvalue=trim(fieldvalue,"\"\"") 
| eval _raw=mvzip(fieldname,fieldvalue,"=") 
| kv

This is not valid JSON. spath is not useful. but, we can create valid JSON. As @manjunathmeti did.
I use rex to extract fields and values. and, rename to _raw .
so kv is useful. AS @woodcock did.

0 Karma

kotig
Path Finder

Yea tried this one unfortunately i am getting empty result. Thank you for help..

0 Karma

manjunathmeti
Champion

Try this:

<your search> | rex field=_raw mode=sed "s/This is my logfile ->\smyfulljson\s{\s/{\"/g" | rex field=_raw mode=sed "s/\s{\svalue:/\":/g" | rex field=_raw mode=sed "s/\s}\s/,\"/g" | spath

richgalloway
SplunkTrust
SplunkTrust

Just a guess.

... | eval _raw = replace ( _raw, "This is my logfile ->", "") | spath | ...
---
If this reply helps you, Karma would be appreciated.
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Why Splunk Customers Should Attend Cisco Live 2026 Las Vegas

Why Splunk Customers Should Attend Cisco Live 2026 Las Vegas     Cisco Live 2026 is almost here, and this ...

What Is the Name of the USB Key Inserted by Bob Smith? (BOTS Hint, Not the Answer)

Hello Splunkers,   So you searched, “what is the name of the usb key inserted by bob smith?”  Not gonna lie… ...

Automating Threat Operations and Threat Hunting with Recorded Future

    Automating Threat Operations and Threat Hunting with Recorded Future June 29, 2026 | Register   Is your ...