Splunk Search

How to extract fields from my string logs

kotig
Path Finder

I have something like below logged in as a message.
How can i replace "This is my logfile ** ->" with empty and then how to extract name, startdate, dept, enddate, status, id and get the values.
This is my logfile ** ->
myfulljson {
name {
value: "Test"
}
startdate {
value: "2020-02-21"
}
dept {
value: 110
}
enddate {
value: "20200220"
}
status {
value: "finish"
}
id {
value: "1234"
}
}

0 Karma

woodcock
Esteemed Legend

Your trash log is not even CLOSE to valid JSON.
I used this tool to figure out how to fix it:
https://jsonlint.com/
Then I ended up with this:

| makeresults 
| eval _raw="This is my logfile ->
 myfulljson {
 name {
 value: \"Test\"
     }
 startdate {
 value: \"2020-02-21\"
     }
 dept {
 value: 110
 }
 enddate {
 value: \"20200220\"
     }
 status {
 value: \"finish\"
     }
 id {
 value: \"1234\"
     }
 }"
| rename COMMENT AS "Strip off the header garbage"
| rex mode=sed "s/^This is my logfile\s*\->[\r\n\s]*myfulljson[\r\n\s]*//"
| rename COMMENT AS "Transform the non-JSON to mostly-compliant-JSON"
| rex mode=sed "s/(\w+)([\r\n\s]*\{)/\"\1\":\2/g s/value: /\"value\": /g s/\}([\r\n\s]*\")/},\1/g"
| rename COMMENT AS "OKish JSON is now parseable with 'KV_MODE=json'"
| kv

woodcock
Esteemed Legend

To do this properly, you should transform this garbage into fully-compliant JSON (that means removing ALL whitesapce) BEFORE it is indexed. This will save you license and allow KV_MODE=json to "just work".

0 Karma

to4kawa
Ultra Champion

UPDATE:

| makeresults 
| eval _raw="This is my logfile ->
myfulljson {
name {
value: \"Test\"
}
startdate {
value: \"2020-02-21\"
}
dept {
value: 110
}
enddate {
value: \"20200220\"
}
status {
value: \"finish\"
}
id {
value: \"1234\"
}
}" 
| rex max_match=0 "(?m)(?<fieldname>[a-z]+)\s{\s*value:\s(?<fieldvalue>\S+)" 
| eval fieldvalue=trim(fieldvalue,"\"\"") 
| eval _raw=mvzip(fieldname,fieldvalue,"=") 
| kv

This is not valid JSON. spath is not useful. but, we can create valid JSON. As @manjunathmeti did.
I use rex to extract fields and values. and, rename to _raw .
so kv is useful. AS @woodcock did.

0 Karma

kotig
Path Finder

Yea tried this one unfortunately i am getting empty result. Thank you for help..

0 Karma

manjunathmeti
Champion

Try this:

<your search> | rex field=_raw mode=sed "s/This is my logfile ->\smyfulljson\s{\s/{\"/g" | rex field=_raw mode=sed "s/\s{\svalue:/\":/g" | rex field=_raw mode=sed "s/\s}\s/,\"/g" | spath

richgalloway
SplunkTrust
SplunkTrust

Just a guess.

... | eval _raw = replace ( _raw, "This is my logfile ->", "") | spath | ...
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...