sourcetype question and confusion

[rchittip]

I would like to understand the sourcetype usage scenario in splunk for forwarders, Indexers and search head.

In my environment I have all individual dedicated splunk servers.

Search Head - 2
Indexers (In load balance) -5
Universal Forwarders - 3k

I usually configure sourcetype in forwarders inputs.conf using my deployment server.

Scenario 1:
Using above approach, can I go for the field extraction at search head without creating the same sourcetype on the searchhead?

Scenario 2:
Let's say I have few fields configured with sourcetype on the searchhead. what if the case if I delete the sourcetype on searchhead but the same sourcetype is being forwarded from UF. Should I loose all my field extractions?

Scenario 3:
If I need to do masking on logs, should I have to create same sourcetype on the Indexers?

Request to suggest the best practice in creating the sourcetype in splunk components searchhead , Indexers and forwarder.

Thanks,
Ramu Chittiprolu

[pruthvikrishnap]

Hi Ramu,

if you are planning to rename any sourcetypes on searchhead, or planning to work on differently named source types same at search time below documentation gives you more insight.

https://docs.splunk.com/Documentation/Splunk/7.1.1/Data/Renamesourcetypes

[somesoni2 · Accepted solution]

Here is how sourcetype is used in each of Splunk components

Universal Forwarders - This does monitoring of data and tag the data with the sourcetype defined in the inputs.conf file. Since you're using a sourcetype already, you're good

*Indexers * - With topology of yours, the Indexers will do the data parsing and indexing. The sourcetype definition, as configured in props.conf file on Indexer, will be used to process the events, which includes event breaking, timestamp parsing , index-time field extraction, data masking etc . If you're using a custom sourcetype (not using built-in sourcetypes), then it's recommended that you configure your indexers to have sourcetype definitions. Without this Splunk will use it's default setting to parse the data, which can be, based on your data, can be inefficient.

Search Head - All search-time field extraction for a sourcetype would happen here, so they should be setup here (in props.conf with or without transforms.conf).

Scenario 1:
Using above approach, can I go for the field extraction at search head without creating the same sourcetype on the searchhead?
[Ans: Not sure I get it. A field extraction (on Search Head) is tied to a sourcetype/source/host. So you can create it for source/host without creating stanza for the sourcetype (not sure if this is what you were looking for)]

Scenario 2:
Let's say I have few fields configured with sourcetype on the searchhead. what if the case if I delete the sourcetype on searchhead but the same sourcetype is being forwarded from UF. Should I loose all my field extractions?
[Ans: If Field extraction are setup at sourcetype level on Search Head, you'll loose them when you delete them]

Scenario 3:
If I need to do masking on logs, should I have to create same sourcetype on the Indexers?
[Ans: As mentioned before, data masking happens at Indexer level, so you've create necessary configuration on indexer to do your masking]

[ddrillic]

What are you trying to achieve?

-- Using above approach, can I go for the field extraction at search head without creating the same sourcetype on the searchhead?