Getting Data In

Regexp for transform.conf doesnt work

evkuzin
New Member

Splunk receive a log like this:

Nov 15 13:02:10 172.20.20.3 test WARNING 1 "Invalid path" 178.217.60.3 0 10.18.7.98 2040 5 "bla bla bla" sampled 1 0 N/A low drop FFFFFFFF-FFFF-FFFF-000E-000059C98546

And vendor said that each field is separated by a single space character. Fields that may contain spaces are printed between double quotes. So for each field i use configuration like this:

props.conf:

[test_source]
REPORT-device_ip=device_ip
REPORT-attack_name=attack_name

transforms.conf

[device_ip]
REGEX = (".*?"|\S+)
FORMAT = device_ip::$4
MV_ADD = true

[attack_name]
REGEX = (".*?"|\S+)
FORMAT = attack_name::$8
MV_ADD = true

I've already test regexp with https://regex101.com/ and it should just split fields in accordance to vendors documentation, but it doesn't work.

all files I put in /opt/splunk/etc/system/local/

0 Karma

FrankVl
Ultra Champion

Your regex contains just 1 capturing group, so how can you refer to $4 and $8?

Yes, this regex matches all the bits separated by spaces, but this is not how you extract fields from an event like this.

Approaches that would work:

REGEX = (".*?"|\S+)\s(".*?"|\S+)\s(".*?"|\S+)\s etc.
FORMAT = field1::$1 field2::$2 field3::$3 etc.

Which uses a regex that matches the event as a whole and extracts each field into a capture group and then assigns those capture groups to the relevant fieldname.

Or:

REGEX = (?:(?:".*?"|\S+)\s){10}(".*?"|\S+)
FORMAT = ip::$1

Which skips over X (10 in this example) fields and then extracts a single piece to assign to a specific field.

0 Karma

joebisesi
Path Finder

Have you tried setting the MV_Add to its default value? Which is false.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The second IP address ("defence_attack_name") is group #9, not 8.
When you say "it doesn't work", what exactly do you mean? What results are you getting?

---
If this reply helps you, Karma would be appreciated.
0 Karma

evkuzin
New Member

Actually I'm getting nothing 🙂
I mean that nothing extracted at all except field vendor_product which defined as EVAL-vendor_product = "Radware" in props.conf
You can see it on screenshot https://ibb.co/d6qhoR
How could I debug fields extraction?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...