topic Re: Where should I apply props and transforms: search heads or indexers? in Getting Data In

Where should I apply props and transforms: search heads or indexers?

rrussellstscied — Fri, 16 Dec 2016 15:17:53 GMT

I thought I had this figured out but am not so certain now.

I need to apply a props and transform to some of our logs to make them readable since they are in a custom format. Should this be sent to the indexers, we have clustered indexers or should they be sent to the search heads?

I believe its the indexers so that the data can be extracted at search time. Please set me straight.

Thanks
Ron

conf files below in case it would help.

Props.conf -

[source::.../dads_logs/*.log]
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE_DATE=true
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD = 23
REPORT-dads_extractions = extract_dads, extract_dads_keywords
TZ = UTC
EXTRACT-filename_for_dms = \/(?\w+\.log) in source

Transform.conf -

[extract_dads]
REGEX= (?[^\s]+)\s+(?[^\s]+)\s+(?[^\s]+)\s+(?[^\s]+)\s+(?[^\s]+)\s+(?[^\s]+)\s+\[(?[^\]]+)\]\s+\[(?[^\]]+)\]

[extract_dads_keywords]
SOURCE_KEY = dads_keywords
REGEX = ,([^,]+)
MV_ADD = true

[dms_host_staging_lookup]
filename = dms_host_staging_lookup.csv

Re: Where should I apply props and transforms: search heads or indexers?

richgalloway — Fri, 16 Dec 2016 16:47:14 GMT

For index-time extractions, put the transforms on the indexers. For search-time extractions, put them on the search heads.

Re: Where should I apply props and transforms: search heads or indexers?

rrussellstscied — Fri, 16 Dec 2016 16:51:29 GMT

Thanks, its very clear to me now.

Re: Where should I apply props and transforms: search heads or indexers?

aaraneta_splunk — Fri, 16 Dec 2016 19:45:37 GMT

@rrussellstsciedu - If richgalloway was able to clarify and answer your question, please don't forget to click "Accept" below his answer to resolve this post. Thanks!

Re: Where should I apply props and transforms: search heads or indexers?

potnuru — Thu, 18 Jun 2020 15:49:47 GMT

@richgalloway What if it is the Standalone Installation of Splunk. I mean Search Head and the Indexer are the same?

Re: Where should I apply props and transforms: search heads or indexers?

richgalloway — Thu, 18 Jun 2020 15:58:02 GMT

On standalone installations of Splunk, the indexer and search head are the same so there's only one location for configurations.

Re: Where should I apply props and transforms: search heads or indexers?

potnuru — Mon, 22 Jun 2020 11:58:28 GMT

@richgalloway

Yes, On standalone installations of Splunk there's only one location for configurations. But I want to understand where those configs are applied on data before Indexing(Index Time) it are after after Indexing(Search Time).

So Basically, I want to know how can we differentiate index-time extractions and search-time extractions.