Getting Data In

Where should I apply props and transforms: search heads or indexers?

I thought I had this figured out but am not so certain now.

I need to apply a props and transform to some of our logs to make them readable since they are in a custom format. Should this be sent to the indexers, we have clustered indexers or should they be sent to the search heads?

I believe its the indexers so that the data can be extracted at search time. Please set me straight.

Thanks
Ron

conf files below in case it would help.

Props.conf -

[source::.../dads_logs/*.log]
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE_DATE=true
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD = 23
REPORT-dads_extractions = extract_dads, extract_dads_keywords
TZ = UTC
EXTRACT-filename_for_dms = \/(?\w+\.log) in source

Transform.conf -

[extract_dads]
REGEX= (?[^\s]+)\s+(?[^\s]+)\s+(?[^\s]+)\s+(?[^\s]+)\s+(?[^\s]+)\s+(?[^\s]+)\s+\[(?[^\]]+)\]\s+\[(?[^\]]+)\]

[extract_dads_keywords]
SOURCE_KEY = dads_keywords
REGEX = ,([^,]+)
MV_ADD = true

[dms_host_staging_lookup]
filename = dms_host_staging_lookup.csv
0 Karma
1 Solution

SplunkTrust
SplunkTrust

For index-time extractions, put the transforms on the indexers. For search-time extractions, put them on the search heads.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

SplunkTrust
SplunkTrust

For index-time extractions, put the transforms on the indexers. For search-time extractions, put them on the search heads.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

Path Finder

@richgalloway What if it is the Standalone Installation of Splunk. I mean Search Head and the Indexer are the same?

0 Karma

SplunkTrust
SplunkTrust
On standalone installations of Splunk, the indexer and search head are the same so there's only one location for configurations.
---
If this reply helps you, an upvote would be appreciated.
0 Karma

Path Finder

@richgalloway 

Yes, On standalone installations of Splunk there's only one location for configurations. But I want to understand where those configs are applied on data before Indexing(Index Time) it are after after Indexing(Search Time).

So Basically, I want to know how can we differentiate index-time extractions and search-time extractions.

0 Karma

Thanks, its very clear to me now.

0 Karma

Splunk Employee
Splunk Employee

@rrussellstsciedu - If richgalloway was able to clarify and answer your question, please don't forget to click "Accept" below his answer to resolve this post. Thanks!

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!