Getting Data In

How to route to index by Source

danielrichards
Explorer

Hi All,

Having an issue trying to route events to an index by source, posting as a new question as I've not found anything that's helped me understand how /where to configure this.

We have events being streamed to HEC (Token) hosted on a HF, which is then forwarding the events to an Indexer, all events are ending up in the Main index on the Indexer.

How can events of the default field Source 'xyz' be sent to a specific Indexer Index 'index_xyz'?


I've seen numerous posts about routing to a specific Index using the SourceType but not Source. I know props.conf and transforms.conf are needed but I've not seen any examples for using Source, also I'm unsure whether they should be implemented on the HF or the Indexer...

The resoning for using Source for routing to a specific index is that these events are always lsted as the Token Name 'xyz'.

TIA

Daniel

0 Karma
1 Solution

gcusello
Legend

Hi @danielrichards,

the approach to indexes choice should be: to put in the same index events with the same retention and the same access rules.

It isn't so relevant but it isn't i a good idea to put different sources in different indexes because you'll have to manage more indexes than required.

If you want to do this, you can override the index in two ways:

Ciao.

Giuseppe

View solution in original post

gcusello
Legend

Hi @danielrichards,

the approach to indexes choice should be: to put in the same index events with the same retention and the same access rules.

It isn't so relevant but it isn't i a good idea to put different sources in different indexes because you'll have to manage more indexes than required.

If you want to do this, you can override the index in two ways:

Ciao.

Giuseppe

danielrichards
Explorer

Hi Giuseppe,

I had tried to specify the index in the HEC Inputs without success.  Not sure why.

Following the instructions at https://community.splunk.com/t5/Getting-Data-In/How-can-I-override-an-index-name-based-on-sourcetype... . worked, thanks.

Interestingly it only works when defining the props.conf & transform.conf on the HF, and not the Indexer...

0 Karma

gcusello
Legend

Hi @danielrichards,

you have to make the overriding on the first Splunk full system (not UFs) that cooks the ingested logs.

In other words: if you have an HF you have to put conf files on it, except if you don't send cooked logs, in this case you have to put them on Indexers.

I usually put these conf files both on HFs and Indexers to be more sure.

Ciao.

Giuseppe

P.S.: Karma Points are appreciated by all the Contributors. 😉

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

And use source instead of sourcetype is shown here: https://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides

With HEC if you have several tokens based on source then the easiest way is set it in inputs.conf as @gcusello said.

r. Ismo

*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>