Knowledge Management

data pipeline and configuration files location

ahmedragy922
Explorer

Hello,
i'm confused about where configuration files (Search Head or Indexer) should i modify when i want to do filed extraction ??
or when i want to override sourcetype,source,host , should i do that in forwarder or indexer or search head???
is there any reference that map the configuration files to which data pipeline applies ?? for example : if i want to do field extraction >>> i should do that in Search head and configure props.conf and transforms.conf

i just found those 2 articles but i still confused.
https://docs.splunk.com/Documentation/Splunk/7.2.6/Admin/Configurationparametersandthedatapipeline
https://docs.splunk.com/Documentation/Splunk/7.2.6/Deploy/Datapipeline

Tags (1)
1 Solution

richgalloway
SplunkTrust
SplunkTrust

You might find this page useful: https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F
Index-time field extraction (fields that will be stored in the indexes) go in heavy forwarders or indexers, whichever touches the data first.
Search-time field extractions (those done during a search) go in search heads.
Overrides of sourcetype, source, or host go in heavy forwarders or indexers, whichever touches the data first.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

You might find this page useful: https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F
Index-time field extraction (fields that will be stored in the indexes) go in heavy forwarders or indexers, whichever touches the data first.
Search-time field extractions (those done during a search) go in search heads.
Overrides of sourcetype, source, or host go in heavy forwarders or indexers, whichever touches the data first.

---
If this reply helps you, Karma would be appreciated.

ahmedragy922
Explorer

thank you for the answer , but i think i can override sourcetype,index,source and host in inputs.conf in Universal Forwarder , also i can do the same in indexer and Heavy Forwarder.
but i think there is the difference between them , in Universal Forwarder i can just write the index where the data will be stored in indexer but i don't have any power to filter the data as in inputs level splunk can't determine the events. in the opposite in indexer , the splunk can parse the data so i can dynamically override (writing regex to change a subset of data or routing some data to index and other to another index) the sourcetype,index,host,source for the data .
can you correct me if i'm wrong ??

richgalloway
SplunkTrust
SplunkTrust

One can specify sourcetype, index, source, and host in a UF, but since that where the data originates, I wouldn't call it an "override". The rest of your statement is correct.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In December, the Splunk Threat Research Team had 1 release of new security content via the Enterprise Security ...

Why am I not seeing the finding in Splunk Enterprise Security Analyst Queue?

(This is the first of a series of 2 blogs). Splunk Enterprise Security is a fantastic tool that offers robust ...

Index This | What are the 12 Days of Splunk-mas?

December 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...