Getting Data In

Regexp for transform.conf doesnt work

evkuzin
New Member

Splunk receive a log like this:

Nov 15 13:02:10 172.20.20.3 test WARNING 1 "Invalid path" 178.217.60.3 0 10.18.7.98 2040 5 "bla bla bla" sampled 1 0 N/A low drop FFFFFFFF-FFFF-FFFF-000E-000059C98546

And vendor said that each field is separated by a single space character. Fields that may contain spaces are printed between double quotes. So for each field i use configuration like this:

props.conf:

[test_source]
REPORT-device_ip=device_ip
REPORT-attack_name=attack_name

transforms.conf

[device_ip]
REGEX = (".*?"|\S+)
FORMAT = device_ip::$4
MV_ADD = true

[attack_name]
REGEX = (".*?"|\S+)
FORMAT = attack_name::$8
MV_ADD = true

I've already test regexp with https://regex101.com/ and it should just split fields in accordance to vendors documentation, but it doesn't work.

all files I put in /opt/splunk/etc/system/local/

0 Karma

FrankVl
Ultra Champion

Your regex contains just 1 capturing group, so how can you refer to $4 and $8?

Yes, this regex matches all the bits separated by spaces, but this is not how you extract fields from an event like this.

Approaches that would work:

REGEX = (".*?"|\S+)\s(".*?"|\S+)\s(".*?"|\S+)\s etc.
FORMAT = field1::$1 field2::$2 field3::$3 etc.

Which uses a regex that matches the event as a whole and extracts each field into a capture group and then assigns those capture groups to the relevant fieldname.

Or:

REGEX = (?:(?:".*?"|\S+)\s){10}(".*?"|\S+)
FORMAT = ip::$1

Which skips over X (10 in this example) fields and then extracts a single piece to assign to a specific field.

0 Karma

joebisesi
Path Finder

Have you tried setting the MV_Add to its default value? Which is false.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The second IP address ("defence_attack_name") is group #9, not 8.
When you say "it doesn't work", what exactly do you mean? What results are you getting?

---
If this reply helps you, Karma would be appreciated.
0 Karma

evkuzin
New Member

Actually I'm getting nothing 🙂
I mean that nothing extracted at all except field vendor_product which defined as EVAL-vendor_product = "Radware" in props.conf
You can see it on screenshot https://ibb.co/d6qhoR
How could I debug fields extraction?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) v3.54.0

The Splunk Threat Research Team (STRT) recently released Enterprise Security Content Update (ESCU) v3.54.0 and ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...