topic Re: Splunk regex ignore fields before match in Splunk Search

Splunk regex ignore fields before match

leandromatperei — Thu, 06 Jan 2022 18:26:57 GMT

I need to extract the contents of the message field into a json log, but the first strings must be ignored until 'stdout F', I can only get the one in front, the second timestamp
Any ideas how to do this?

Examples:

{ app: app01 message: 2022-01-06T17:57:25.799919642Z stdout F [2022-01-06 09:00:00,799] INFO - INFO region: southamerica-east1 } { app: app02 message: 2022-01-06T17:57:25.799919642Z stdout F [2022-01-06 10:20:25,799] ERROR - APIAuthenticationHandler API authentication failure region: southamerica-east1 } { app: app03 message: 2022-01-06T17:57:25.799919642Z stdout F [2022-01-06 12:57:00,799] WARN - failure due to Invalid Credentials region: southamerica-east1 } { app: app04 message: 2022-01-06T17:57:25.799919642Z stdout F [2022-01-06 14:57:25,799] WARN - APIAuthenticationHandler API authentication region: southamerica-east1 }

Re: Splunk regex ignore fields before match

yuanliu — Thu, 06 Jan 2022 18:52:16 GMT

Are you trying to set indexing-time extraction? You can force timestamp format in Splunk Web or TIME_FORMAT in props.conf for that sourcetype; the format would be "[%Y-%m-%d %H:%M:%S,%3N]" in your case.

Re: Splunk regex ignore fields before match

leandromatperei — Thu, 06 Jan 2022 19:05:25 GMT

I can't change the props level, it would have to be in the search itself.

Re: Splunk regex ignore fields before match

ITWhisperer — Fri, 07 Jan 2022 08:08:16 GMT

Try this

| rex "message.*stdout F (?<message>.*)"

Re: Splunk regex ignore fields before match

yuanliu — Fri, 07 Jan 2022 08:17:37 GMT

In this case, this should work

| rex field=message "stdout F \[(?<time2>[^\]]+)" | convert timeformat="%Y-%m-%d %H:%M:%S,%3N" mktime(time2)

After convert, time2 will now contain epoch value. If your intention is to use it as event _time, you can do

| rex field=message "stdout F \[(?<time2>[^\]]+)" | convert timeformat="%Y-%m-%d %H:%M:%S,%3N" mktime(time2) as _time