Splunk Search

How can we perform a lookup substitution at index time?

ddrillic
Ultra Champion

How can we perform a lookup substitution at index time? We have a defined lookup and at index time we would like to replace certain values with the values in the lookup table.

Tags (1)
0 Karma
1 Solution

mayurr98
SplunkTrust
SplunkTrust

hello @ddrillic

You probably may have found out by now but just in case .. Lookups cannot be done at index time but only at search time.
Refer this answers that I just found out
https://answers.splunk.com/answers/8087/kicking-off-lookup-at-index-time.html
https://answers.splunk.com/answers/13723/large-table-lookup-at-index-time-vs-search-time-tradeoffs.h...

Well, you can configure automatic lookups.
let me know if this helps!

View solution in original post

ejwade
Communicator

I was looking to do the same thing, and noticed this doc page was created for 8.1.x.

https://docs.splunk.com/Documentation/Splunk/8.1.3/Data/IngestLookups

Maybe something to look at?

0 Karma

mayurr98
SplunkTrust
SplunkTrust

hello @ddrillic

You probably may have found out by now but just in case .. Lookups cannot be done at index time but only at search time.
Refer this answers that I just found out
https://answers.splunk.com/answers/8087/kicking-off-lookup-at-index-time.html
https://answers.splunk.com/answers/13723/large-table-lookup-at-index-time-vs-search-time-tradeoffs.h...

Well, you can configure automatic lookups.
let me know if this helps!

View solution in original post

ddrillic
Ultra Champion

Very kind @mayurr98 - thanks.

0 Karma

livehybrid
Contributor

Hi, by specifying OUTPUT as part of your lookup command, it will overwrite fields in your results with the value from the lookup if the fields match. e.g:

sourcetype=access_* | stats count by status | lookup status_desc status OUTPUT description

In this example, any previous description field will be overwritten.

However, if the field in your event is called myDescription then you would use:

sourcetype=access_* | stats count by status | lookup status_desc status OUTPUT description AS myDescription

I hope this helps.

ddrillic
Ultra Champion

Great, but we would like to do it at index time ; -)

0 Karma

livehybrid
Contributor

Whoops - Should have read more carefully! Sorry but that is a bit trickier. Its not possible to do a traditional lookup. You're best bet would probably be a time-based lookup so your lookup at searchtime is accurate to the time the data was indexed...it depends on your specific case.
Sorry I couldnt help further!

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!