topic Fields extraction in file with single and multiline logs in Splunk Search

Fields extraction in file with single and multiline logs

clementros — Wed, 13 Feb 2019 15:41:20 GMT

Hi,

I'm new in splunk. I currently want to analyse a log file. I'm facing issues to extract informations because each lines contain different informations. The file contain single and multiline logs.

Here is an example of the log file :

I want to extract generale information like the timestamp (2019-02-08 00:40:03) and the status (INFO|TRACE or other message).

And i also want to be able to extract other infromations in each log line.

Thanks for your help

Re: Fields extraction in file with single and multiline logs

woodcock — Thu, 14 Feb 2019 00:57:13 GMT

If the log file was onboarded properly, Splunk will be breaking the file into events every time there is a timestamp at the beginning of the line and this timestamp will be put into the _time field. So there is no reason to extract the timestamp again. If you need it presented in a different way, just do | eval timestamp=strftime(_time, "Your Time Format Here") To get the other fields, you just need to create appropriate RegEx strings. For example, you can do this:

| . rex "[\r\n\s]+\[TRACE]\s+Reason\s+{\s+(?<Reason>[^\r\n]+)"

Re: Fields extraction in file with single and multiline logs

clementros — Thu, 14 Feb 2019 07:59:12 GMT

Thanks for your answer 🙂

So i need to build a regex for every field extraction ?

Re: Fields extraction in file with single and multiline logs

woodcock — Thu, 14 Feb 2019 08:03:02 GMT

Either that or you need to clip out all of the extra junk to reconstruct the original JSON and pass it to spath.