Luxembourg User Group

Is it possible to create an if/else statement with the TRANSFORMS_ field into props.conf file ?

Zanusha443
Explorer

I have data coming from a single source

but I want to send the events that match a REGEX to an index and all the other that not match it to another index.


I have already tried to change the order of the fields into the TRANSFORMS_ but it still put the events in both indexes.
This is the content of the props.conf file:

[tmpproxy]

TRANSFORMS_routing1 =CIDR_Routing_matched, CIDR_Routing_others

and this is the content of the transforms.conf file:

[CIDR_Routing_matched]
REGEX =src_host\=(?:10\.10\.10\.\d{1,3}|)
FORMAT = tmp_matched_proxy
DEST_KEY=_MetaData:Index
WRITE_META=true

[CIDR_Routing_others]
REGEX =.+
FORMAT = tmp_others_proxy
DEST_KEY=_MetaData:Index
WRITE_META=true

Is it possible to stop the TRANSFORMS_ field in the props.conf file after the first good match?

Labels (3)
0 Karma
1 Solution

Zanusha443
Explorer

At the end I solved the issue with the help of the global community:
https://community.splunk.com/t5/Getting-Data-In/Is-it-possible-to-create-an-if-else-statement-with-t...

 

Thanks for the help.

Angelo

View solution in original post

0 Karma

Zanusha443
Explorer

At the end I solved the issue with the help of the global community:
https://community.splunk.com/t5/Getting-Data-In/Is-it-possible-to-create-an-if-else-statement-with-t...

 

Thanks for the help.

Angelo

0 Karma

PickleRick
SplunkTrust
SplunkTrust

What do you mean by "it still puts the events in both indexes". This is a very unexpected thing since you can _route_ a single event into one index or another but the typical problem is quite the oposite - you actually can't get an event into two separate indexes unless you explicitly clone it in your processing pipeline with CLONE_SOURCETYPE.

Zanusha443
Explorer

You are right, checking it again there are no duplicate events over the 2 indexes.

0 Karma

PickleRick
SplunkTrust
SplunkTrust

OK. So we got this one out of the way 🙂

Now to the point - there is no way to "conditionally" apply a transform or not depending on some external condition. You could use indexed fields to extract some parameters during ingestion pipeline and then unset them so they don't get indexed but that would be completely unmaintainable and plain ugly.

Normally the trick is to order your transforms properly so that if an event matches several of those transforms, the last one is the relevant one. (similar to "ACL" concept but kinda in reverse)

0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @Zanusha443,

Can you try changing the order like below? 

TRANSFORMS_routing1 = CIDR_Routing_others, CIDR_Routing_matched
If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma
Get Updates on the Splunk Community!

Monitoring Postgres with OpenTelemetry

Behind every business-critical application, you’ll find databases. These behind-the-scenes stores power ...

Mastering Synthetic Browser Testing: Pro Tips to Keep Your Web App Running Smoothly

To start, if you're new to synthetic monitoring, I recommend exploring this synthetic monitoring overview. In ...

Splunk Edge Processor | Popular Use Cases to Get Started with Edge Processor

Splunk Edge Processor offers more efficient, flexible data transformation – helping you reduce noise, control ...