Getting Data In

Applying props on UF and transforms on the Indexer in Splunk 6

theouhuios
Motivator

Hello

I am trying to get the IIS data from windows hosts and it looks like we can apply the props.conf on the UF itself. But then I also want to apply transforms which dont work on the UF on the indexer for that sourcetype. Would it be possible to call the Transforms from the UF props itself? Or do I need props to go to both the UF and Indexers but transforms to go to just the indexers? From my understanding if the filtering and indexing is happening at UF using the props then the indexers will not try to index it again,right? If that's true how will it apply the transforms?

props

[iis]
TRANSFORMS-source_extraction = w3svc_name

transforms

[w3svc_name]
SOURCE_KEY = MetaData:Source
DEST_KEY   = MetaData:Source
REGEX      = (?i)\\(W3SVC[^\\]*)
FORMAT     = source::$1
Tags (3)
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

First note it's TRANSFORMS-class, you're missing an S there.
Second, I believe those need to go on the indexer for parsing, a UF doesn't do that phase: http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

For rewriting the source field I'm using these without WRIE_META:

SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Source 

Additionally, it seems as if your REGEX and FORMAT values don't really work together. Your capturing group 1 is either "es" or "gs", I guess that's not the intended source value.

0 Karma

theouhuios
Motivator

Also when I try it this way it doesn't work

[host::*WN*]
TRANSFORMS-source_extraction = source_ext

[source_ext]
SOURCE_KEY = field:source
REGEX = (es|gs)\S{1}(?P<source>[A-Z0-9]{6,7})
FORMAT = source::$1
WRITE_META = true`
0 Karma

theouhuios
Motivator

Tried this. The extraction works when i give sourcetype as microsoft_iis but not when I give sourcetype as iis. I guess thats because its a pre defined sourcetype. But the Indexed_extractions = w3c doesn't parse the fields properly when I use any sourcetype other than iis.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

The UF can and will handle a props.conf file, but it won't be able to use every single setting from it.

By "those" I'm referring to TRANSFORMS-class settings in props.conf - the transforms.conf needs to be on the indexer(s) entirely.
You can have a full copy of props.conf on both the UF and indexer(s), they will pick out the settings they can use.

theouhuios
Motivator

Missed S while pasting it here. When you say those you mean both props and transforms? Someone from Splunk actually told us that UF can handle props and it looks like it does too as it lists the props in btool.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Community Content Calendar, September edition

Welcome to another insightful post from our Community Content Calendar! We're thrilled to continue bringing ...

Splunkbase Unveils New App Listing Management Public Preview

Splunkbase Unveils New App Listing Management Public PreviewWe're thrilled to announce the public preview of ...

Leveraging Automated Threat Analysis Across the Splunk Ecosystem

Are you leveraging automation to its fullest potential in your threat detection strategy?Our upcoming Security ...