Splunk Search

Is the automatic lookup table used by the indexer?

rxdeleon
Explorer

When an automatic lookup table is defined, is that used by the indexer to add the new fields or is it the search head that does that?

Tags (1)
0 Karma
1 Solution

skoelpin
SplunkTrust
SplunkTrust

It's the search head. The data is already indexed on the indexer so it would be a search time function

View solution in original post

0 Karma

skoelpin
SplunkTrust
SplunkTrust

It's the search head. The data is already indexed on the indexer so it would be a search time function

0 Karma

rxdeleon
Explorer

Thanks, skoelpin, for the quick reply. If that's the case, does that mean that the raw data found by the indexer would all be shipped to the search head? And that's where the lookup table would be applied?

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Partially correct. The SH will search the data on the indexer, the indexer will not ship its data to the SH

Data lives on the indexer and when a scheduled/ad-hoc search kicks off on the SH, the SH will search the data on the indexers and the automatic lookup logic will be applied at search time. A good way to think about this is, say you create an automatic lookup and want to change it after a day. You can easily change it because it's done on the fly at search time without baking any rules onto the indexers

0 Karma

skoelpin
SplunkTrust
SplunkTrust

@rxdeleon please accept the answer if I answered your question

0 Karma

rxdeleon
Explorer

@skoelpin, I understand that the automatic lookup logic will be applied at search time. But which component does that? Is it the search head or the indexer? If it's the search head, then that means the search results, no matter how big, would be sent to the search head where the automatic lookup logic would be applied.

I would wish that it's the indexer that does it so that extracted fields could be used to filter out irrelevant events to minimize data being sent back to the search head (for performance reasons).

0 Karma

skoelpin
SplunkTrust
SplunkTrust

It's the search head. Lookups have always been a bottleneck which is why I always tell customers that you should use a stats before the lookup.

For index time lookups, you should check out Cribl. It integrates directly with Splunk! I had a long conversation with their CEO @clintsharp at CONF and was pretty impressed with the features it has.

https://blog.cribl.io/2018/09/17/enriching-data-in-motion-with-ingest-time-lookups/

0 Karma

rxdeleon
Explorer

Thanks for the Cribl info, @skoelpin. I'll check it out.

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...