Ignore automatic lookup just for a search

[marcoscala]

Hi!

do you think if there's a way to say Splunk to ignore automatic lookups just for a search? I'm configuring some custom reports on a Splunk installation with ES and PCI Apps. Those apps do an intensive usage of automatic lookups which are fine, but that introduce some overhead. Running my custom report I see from Search Inspector that the most time (28secs out of 31secs)is spent in command.search.lookups.

Is there a way to say: for this seach ignore automatic lookups?

Thanks a lot!

[JimGat_SSI]

Did you ever get an answer to this? I am wanting to disable the automatic lookups for specific searches using triggered fields.

[lukejadamec]

Automatic lookup are used when the search includes the lookup output. For example: If you have an input RETURNCODE and an output returncode_name, and you run a search like this:

search |table RETURNCODE

The cost for command.search.lookups will be zero.

If you run the search:

search |table returncode_name

The cost for command.search.lookups will have a value.

What you'll need to do is create your custom search so that it uses the inputs to the lookup, and not the outputs.

[millern4]

Although this thread is quite old, I'd still like to see if anyone out there is still experiencing this exact same issue?

We've been working with support for some time now to troubleshoot poor search performance on our (4) ES SH Cluster where a majority of our time is spent in command.search.lookups, where in our (6) Non ES SH Cluster running the exact same search produces lighting fast results.

We are obviously aware of the nature of how ES functions with the automatic lookups, but I"m curious as others experience with Enterprise Security searching vs non ES Search performance? Is it similar, better, worse, and how close is the experience between the environments for end users?

Thanks

[lukejadamec]

If this answer helped, please mark it as excepted.

[lukejadamec]

You can lead a horse to water....
Look, when you search for the sourcetype the search will of course find all fields and all associated look ups.

Like I said, you need to include in your search only fields that are not lookup fields.
Try this with your search and check the cost (this is very basic example):
sourcetype=vanguard |fields index

[marcoscala]

I'm sorry to contraddict your reply: in the Splunk for PCI app, for instance, everytime there's a host, src or dest field, the automatic lookups are fired to produce more infos from the assets.csv file. So the cost of in my case has always a quite significant value.

The search "sourcetype=vanguard" in the last 15m has the following costs:
1.959 events in 8,65 seconds.
5,021 command.search
4,049 command.search.lookups