topic Re: How can we perform a lookup substitution at index time? in Splunk Search

How can we perform a lookup substitution at index time?

ddrillic — Wed, 07 Mar 2018 23:36:56 GMT

How can we perform a lookup substitution at index time? We have a defined lookup and at index time we would like to replace certain values with the values in the lookup table.

Re: How can we perform a lookup substitution at index time?

livehybrid — Wed, 07 Mar 2018 23:42:28 GMT

Hi, by specifying OUTPUT as part of your lookup command, it will overwrite fields in your results with the value from the lookup if the fields match. e.g:

sourcetype=access_* | stats count by status | lookup status_desc status OUTPUT description

In this example, any previous description field will be overwritten.

However, if the field in your event is called myDescription then you would use:

sourcetype=access_* | stats count by status | lookup status_desc status OUTPUT description AS myDescription

I hope this helps.

Re: How can we perform a lookup substitution at index time?

ddrillic — Wed, 07 Mar 2018 23:44:21 GMT

Great, but we would like to do it at index time ; -)

Re: How can we perform a lookup substitution at index time?

livehybrid — Wed, 07 Mar 2018 23:51:00 GMT

Whoops - Should have read more carefully! Sorry but that is a bit trickier. Its not possible to do a traditional lookup. You're best bet would probably be a time-based lookup so your lookup at searchtime is accurate to the time the data was indexed...it depends on your specific case.
Sorry I couldnt help further!

Re: How can we perform a lookup substitution at index time?

mayurr98 — Thu, 08 Mar 2018 07:58:26 GMT

hello @ddrillic

You probably may have found out by now but just in case .. Lookups cannot be done at index time but only at search time.
Refer this answers that I just found out
https://answers.splunk.com/answers/8087/kicking-off-lookup-at-index-time.html
https://answers.splunk.com/answers/13723/large-table-lookup-at-index-time-vs-search-time-tradeoffs.html

Well, you can configure automatic lookups.
let me know if this helps!

Re: How can we perform a lookup substitution at index time?

ddrillic — Mon, 12 Mar 2018 00:33:45 GMT

Very kind @mayurr98 - thanks.

Re: How can we perform a lookup substitution at index time?

ejwade — Mon, 26 Apr 2021 19:59:43 GMT

I was looking to do the same thing, and noticed this doc page was created for 8.1.x.

https://docs.splunk.com/Documentation/Splunk/8.1.3/Data/IngestLookups

Maybe something to look at?