Splunk Search

Splunk regex ignore fields before match

leandromatperei
Path Finder

I need to extract the contents of the message field into a json log, but the first strings must be ignored until 'stdout F', I can only get the one in front, the second timestamp
Any ideas how to do this?

Examples:

 

{ 
   app: app01
   message: 2022-01-06T17:57:25.799919642Z stdout F [2022-01-06 09:00:00,799]  INFO - INFO
   region: southamerica-east1
}

{ 
   app: app02
   message: 2022-01-06T17:57:25.799919642Z stdout F [2022-01-06 10:20:25,799]  ERROR - APIAuthenticationHandler API authentication failure
   region: southamerica-east1
}

{ 
   app: app03
   message: 2022-01-06T17:57:25.799919642Z stdout F [2022-01-06 12:57:00,799]  WARN - failure due to Invalid Credentials
   region: southamerica-east1
}

{ 
   app: app04
   message: 2022-01-06T17:57:25.799919642Z stdout F [2022-01-06 14:57:25,799]  WARN - APIAuthenticationHandler API authentication
   region: southamerica-east1
}

 

Labels (1)
Tags (1)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Try this

| rex "message.*stdout F (?<message>.*)"
0 Karma

yuanliu
SplunkTrust
SplunkTrust

Are you trying to set indexing-time extraction?  You can force timestamp format in Splunk Web or TIME_FORMAT in props.conf for that sourcetype; the format would be "[%Y-%m-%d %H:%M:%S,%3N]" in your case.

0 Karma

leandromatperei
Path Finder

I can't change the props level, it would have to be in the search itself.

0 Karma

yuanliu
SplunkTrust
SplunkTrust

In this case, this should work

| rex field=message "stdout F \[(?<time2>[^\]]+)"
| convert timeformat="%Y-%m-%d %H:%M:%S,%3N" mktime(time2)

After convert, time2 will now contain epoch value.  If your intention is to use it as event _time, you can do

| rex field=message "stdout F \[(?<time2>[^\]]+)"
| convert timeformat="%Y-%m-%d %H:%M:%S,%3N" mktime(time2) as _time

 

0 Karma
Get Updates on the Splunk Community!

Splunk Security Content for Threat Detection & Response, Q1 Roundup

Join Principal Threat Researcher, Michael Haag, as he walks through:An introduction to the Splunk Threat ...

Splunk Life | Happy Pride Month!

Happy Pride Month, Splunk Community! &#x1f308; In the United States, as well as many countries around the ...

SplunkTrust | Where Are They Now - Michael Uschmann

The Background Five years ago, Splunk published several videos showcasing members of the SplunkTrust to share ...