Splunk Search

How to extract fields from my string logs

kotig
Path Finder

I have something like below logged in as a message.
How can i replace "This is my logfile ** ->" with empty and then how to extract name, startdate, dept, enddate, status, id and get the values.
This is my logfile ** ->
myfulljson {
name {
value: "Test"
}
startdate {
value: "2020-02-21"
}
dept {
value: 110
}
enddate {
value: "20200220"
}
status {
value: "finish"
}
id {
value: "1234"
}
}

0 Karma

woodcock
Esteemed Legend

Your trash log is not even CLOSE to valid JSON.
I used this tool to figure out how to fix it:
https://jsonlint.com/
Then I ended up with this:

| makeresults 
| eval _raw="This is my logfile ->
 myfulljson {
 name {
 value: \"Test\"
     }
 startdate {
 value: \"2020-02-21\"
     }
 dept {
 value: 110
 }
 enddate {
 value: \"20200220\"
     }
 status {
 value: \"finish\"
     }
 id {
 value: \"1234\"
     }
 }"
| rename COMMENT AS "Strip off the header garbage"
| rex mode=sed "s/^This is my logfile\s*\->[\r\n\s]*myfulljson[\r\n\s]*//"
| rename COMMENT AS "Transform the non-JSON to mostly-compliant-JSON"
| rex mode=sed "s/(\w+)([\r\n\s]*\{)/\"\1\":\2/g s/value: /\"value\": /g s/\}([\r\n\s]*\")/},\1/g"
| rename COMMENT AS "OKish JSON is now parseable with 'KV_MODE=json'"
| kv

woodcock
Esteemed Legend

To do this properly, you should transform this garbage into fully-compliant JSON (that means removing ALL whitesapce) BEFORE it is indexed. This will save you license and allow KV_MODE=json to "just work".

0 Karma

to4kawa
Ultra Champion

UPDATE:

| makeresults 
| eval _raw="This is my logfile ->
myfulljson {
name {
value: \"Test\"
}
startdate {
value: \"2020-02-21\"
}
dept {
value: 110
}
enddate {
value: \"20200220\"
}
status {
value: \"finish\"
}
id {
value: \"1234\"
}
}" 
| rex max_match=0 "(?m)(?<fieldname>[a-z]+)\s{\s*value:\s(?<fieldvalue>\S+)" 
| eval fieldvalue=trim(fieldvalue,"\"\"") 
| eval _raw=mvzip(fieldname,fieldvalue,"=") 
| kv

This is not valid JSON. spath is not useful. but, we can create valid JSON. As @manjunathmeti did.
I use rex to extract fields and values. and, rename to _raw .
so kv is useful. AS @woodcock did.

0 Karma

kotig
Path Finder

Yea tried this one unfortunately i am getting empty result. Thank you for help..

0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

Try this:

<your search> | rex field=_raw mode=sed "s/This is my logfile ->\smyfulljson\s{\s/{\"/g" | rex field=_raw mode=sed "s/\s{\svalue:/\":/g" | rex field=_raw mode=sed "s/\s}\s/,\"/g" | spath

richgalloway
SplunkTrust
SplunkTrust

Just a guess.

... | eval _raw = replace ( _raw, "This is my logfile ->", "") | spath | ...
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

What’s new on Splunk Lantern in August

This month’s Splunk Lantern update gives you the low-down on all of the articles we’ve published over the past ...

Welcome to the Future of Data Search & Exploration

You have more data coming at you than ever before. Over the next five years, the total amount of digital data ...

This Week's Community Digest - Splunk Community Happenings [8.3.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...