Splunk Search

Use Regex to extract the fields

siddharthmis
Explorer

I have data like-

2017-12-19 09:39:41|INFO|4b483c4b138de23b2f83a208c2313c4a|8de3f071aed6401d9ff5c4289694e852|a|b|c
2017-12-19 09:39:41|ERRO|4b483c4b138de23b2f83a208c2313c4a|8de3f071aed6401d9ff5c4289694e852|a|b|c|d

Since there are different number of fields, I am not able to extract these.

^(?<timestamp>[^|]+)\|(?<severity>[^|]+)\|(?<x>[^|]+)\|(?<y>[^|]+)\|(?<a>[^|]+)\|(?<b>[^|]+)\|(?<c>[^|^$]+)\|(?<d>.*) works for one but not the other.
Is there any way I can extract all fields.

0 Karma
1 Solution

horsefez
Motivator

Hey,

look at this:

https://regex101.com/r/reSnrj/1/
(if it doesn't load properly click in the field with the log events first, this will fix it)

^(?:(?<timestamp>[^|]+)\|(?<severity>[^|]+)\|(?<x>[^|]+)\|(?<y>[^|]+)\|(?<a>[^|]+)\|(?<b>[^|]+)\|(?<c>[^|]+)(?:\|(?<d>[^|]+))?)$

View solution in original post

micahkemp
Champion

Assuming your last values (a, b, c...) are of an unknown quantity, you could do something like this:

props.conf:

[<sourcetype>]
REPORT-sourcetypevalues = severityxy,othervalues

transforms.conf:

[severityxy]
REGEX = ^(?<timestamp>[^|]+)\|(?<severity>[^|]+)\|(?<x>[^|]+)\|(?<y>[^|]+)\|(?<othervalues>.*)$

[othervalues]
SOURCE_KEY = othervalues
REGEX = (^|\|)(?<othervalue>[^|]+)($|\|)
MV_ADD = true
0 Karma

horsefez
Motivator

Hey,

look at this:

https://regex101.com/r/reSnrj/1/
(if it doesn't load properly click in the field with the log events first, this will fix it)

^(?:(?<timestamp>[^|]+)\|(?<severity>[^|]+)\|(?<x>[^|]+)\|(?<y>[^|]+)\|(?<a>[^|]+)\|(?<b>[^|]+)\|(?<c>[^|]+)(?:\|(?<d>[^|]+))?)$

cmerriman
Super Champion

This isn't 100% there yet, still noodling, but it's something to start with:

^(?<timestamp>[^|]+)\|(?<severity>[^|]+)\|(?<x>[^|]+)\|(?<y>[^|]+)\|(?<a>[^|]+)\|(?<b>[^|]+)\|(?<c>[^|]+)?[\|]?(?<d>[^|].*)

this should extract them all, but it seems to but c inside d for the event that doesn't have d

0 Karma

Elsurion
Communicator

you can use the same extraction but if the field d is optional you have to define it as optional

(\|(?<d>.*))?

then you can use this regex for both cases.

If you have more optional fields, then you can repeat the optional extraction.

0 Karma

siddharthmis
Explorer

Can someone assist on this?

0 Karma

micahkemp
Champion

Is it always severity, x, y, then a variable number of pipe delimited values afterwards that should all be in one potentially multivalue) field?

0 Karma
Get Updates on the Splunk Community!

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...