Getting Data In

proper way to apply props.conf rules on a modified sourcetype, and other fun stories

yannK
Splunk Employee
Splunk Employee

I tried to setup 2 rules :
- one to rename the sourcetype A to B for some events
- one to apply a SEDCMD rule to the sourcetype B only.

But ultimately the SEDCMD never applied, it works only if I apply i to the sourcetype A.

[A] 
TRANSFORMS-set_sourcetype_to_ssl_audit = force_sourcetype_to_B 

[B] 
SEDCMD-anonymize = s/SSN(\d+)/SSNxxx/g 

transforms.conf

[force_sourcetype_to_B] 
REGEX = BFTW
DEST_KEY = MetaData:Sourcetype 
FORMAT = sourcetype::B 

Is it linked to the order of the rules ?

Tags (2)
1 Solution

yannK
Splunk Employee
Splunk Employee

As per @sowing comment.

The problem is that the 2 rules are for index time.
The first transform will apply renaming the sourcetype A to B.
But the event will not be parsed a second time (at index time) for the new sourcetype rules. Therefore the SEDCMD will never happen.

Remark: at search time. the new sourcetype rules may stilll apply ( by example a field extraction)

View solution in original post

0 Karma

yannK
Splunk Employee
Splunk Employee

As per @sowing comment.

The problem is that the 2 rules are for index time.
The first transform will apply renaming the sourcetype A to B.
But the event will not be parsed a second time (at index time) for the new sourcetype rules. Therefore the SEDCMD will never happen.

Remark: at search time. the new sourcetype rules may stilll apply ( by example a field extraction)

0 Karma

somesoni2
Revered Legend

Per this , both SEDCMD and transforms.conf happen during 'Typing' queue process and since data is coming for sourcetype A so configuration for A will take effect and configuration (index-time) for sourcetype B will never take place.

sowings
Splunk Employee
Splunk Employee

Make that an answer, and I'll vote it up. You get one pass at parsing rules, starting with the original sourcetype, top to bottom, left to right.

0 Karma
Get Updates on the Splunk Community!

Data Preparation Made Easy: SPL2 for Edge Processor

By now, you may have heard the exciting news that Edge Processor, the easy-to-use Splunk data preparation tool ...

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Tips & Tricks When Using Ingest Actions

Tune in to learn about:Large scale architecture when using Ingest ActionsRegEx performance considerations ...