topic Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data? in Splunk Search

How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

dfigurello — Wed, 29 Oct 2014 18:05:09 GMT

Hi everyone,

I need help to create a better regex in my transforms.conf. I am filtering checkpoint data in my Splunk.
In this case, I don't want collect the following event
sourcetype=opsec action=allowed src=172.20.1.1
OR
sourcetype=opsec action=allowed src=172.20.1.2

I created the props.conf and transforms.conf:

props.conf:
[opsec]
TRANSFORMS-t1 = eliminate_opsec 

transforms.conf
[eliminate_opsec]
REGEX = (src\=172.20.1.1|src\=172.20.1.2)
DEST_KEY = queue
FORMAT = nullQueue

I need add in the regex, the field action=allowed.
How do I do add this function ?

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

somesoni2 — Wed, 29 Oct 2014 18:07:09 GMT

Can you post some sample raw data?

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

richgalloway — Wed, 29 Oct 2014 18:13:42 GMT

This matches the examples you gave.

REGEX = action=allowed\s+(src=172.20.1.1|src=172.20.1.2)

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

dfigurello — Mon, 28 Sep 2020 18:02:32 GMT

Sure.

loc=17389746|time=29Oct2014 16:28:39|action=allowed|orig=172.20.1.1|i/f_dir=inbound|i/f_name=eth3|has_accounting=0|product=VPN-1 & FireWall-1|__policy_id_tag=product=VPN-1 & FireWall-1[db_tag={000000E0-003F-0046-93C9-1F533951F91E};mgmt=gerfw;date=1414530153;policy_name=example1]|inzone=Internal|outzone=External|service_id=http|src=172.20.1.1|s_port=58077|dst=173.xxx.yyy.57|service=80|proto=tcp|xlatesrc=172.20.1.xx|xlatesport=29365|xlatedport=0|NAT_rulenum=267|NAT_addtnl_rulenum=1|rule=781

Cheers!

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

dfigurello — Wed, 29 Oct 2014 18:33:22 GMT

Hi Richgalloway,

I tried but not works.

REGEX = action=allowed\s+(src=172.20.1.1|src=172.20.1.2)

and then i tried this:

REGEX = action\=allowed\s+(src\=172.20.1.1|src\=172.20.1.2)

Cheers!

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

somesoni2 — Wed, 29 Oct 2014 19:47:47 GMT

Did 2nd option work?

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

somesoni2 — Wed, 29 Oct 2014 19:50:47 GMT

Try this as well

REGEX = action=allowed(.*)(src=172\.20\.\1\.1|src=172\.20\.1\.2)

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

dfigurello — Wed, 29 Oct 2014 19:51:00 GMT

Hi Somesoni2,

No.
:(

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

dfigurello — Wed, 29 Oct 2014 20:05:07 GMT

Unfortunately no.

When I remove action=allowed, my regex works well.
Any idea?

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

richgalloway — Wed, 29 Oct 2014 20:14:34 GMT

Try this:

REGEX = (action=allowed(.*)(src=172\.20\.\1\.1|src=172\.20\.1\.2))

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

dfigurello — Wed, 29 Oct 2014 22:34:24 GMT

Unfortunately no.

Look above the picture:
Cheers!

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

dfigurello — Wed, 29 Oct 2014 22:35:18 GMT

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

richgalloway — Thu, 30 Oct 2014 12:08:29 GMT

What I see is an event that was indexed because it did not match the regex string in the eliminate_opsec stanza. The match failed because the IP address was not one of the two in the regex. If that is not the expected behavior then please restate the requirements.

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

dfigurello — Thu, 30 Oct 2014 12:43:25 GMT

Hi richgalloway,

It just example, because I can't show real address ip.
I am changing the values (address ip) in transforms.conf, according with requirements.

Tks,

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

richgalloway — Thu, 30 Oct 2014 13:01:26 GMT

I believe there is a stray backslash in the regex string. Try this one:

(action=allowed(.*)(src=172\.20\.1\.1|src=172\.20\.1\.2))

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

dfigurello — Thu, 30 Oct 2014 18:23:43 GMT

Hi Richgalloway!

it worked really well!
Thanks a lot.

Re: How to add two fields using regex in transforms.conf to filter out certain events from checkpoint data?

dfigurello — Thu, 30 Oct 2014 18:24:43 GMT

Thanks a lot too Somesoni2.
:)