https://community.splunk.com/t5/Splunk-Search/Lookups-slow-performance/m-p/397847#M115451 <P>It depends on how many definitions exist to create the field <CODE>guid</CODE> and what types they are. This is a VERY deep topic. Start here and pay special attention to the parts regarding <CODE>lispy</CODE>:<BR /> <A href="https://conf.splunk.com/files/2017/slides/fields-indexed-tokens-and-you.pdf">https://conf.splunk.com/files/2017/slides/fields-indexed-tokens-and-you.pdf</A><BR /> Also check out this:<BR /> <A href="https://splunkbase.splunk.com/app/2871/">https://splunkbase.splunk.com/app/2871/</A></P> <P>If the field <CODE>guid</CODE> is an indexed field, then you can use this:</P> <PRE><CODE>index=my_index guid::my_guid </CODE></PRE> Sun, 03 Mar 2019 05:14:09 GMT woodcock 2019-03-03T05:14:09Z https://community.splunk.com/t5/Splunk-Search/Lookups-slow-performance/m-p/397845#M115449 <P><STRONG>Background</STRONG></P> <UL> <LI>We are a new SplunkCloud customer and are building out our instance, setting up our indexes, field extractions, etc. I’m currently working on Lookups and and seeing unexpected performance characteristics from the searches I am running. </LI> <LI>I created an automatic lookup that links the data in one of our indexes to a lookup table that has about 15k rows and 7 columns of data. The automatic lookup links the index to the lookup table via a “guid" field. </LI> </UL> <P><STRONG>Issue:</STRONG></P> <UL> <LI>This search takes 48 seconds to complete and has a scan count of 16million <CODE>index=my_index guid=my_guid</CODE></LI> <LI>This search takes 300ms to complete and has a scan count of 410 <CODE>index=my_index my_guid</CODE></LI> </UL> <P>Why is the first search doing all of this extra work? We are about to roll out access to Splunk to about 150 employees. I want to make sure I understand the proper way to recommend people to run searches against this index that is linked to the lookup table.</P> <P>Thanks in advance,<BR /> Chris</P> Tue, 26 Feb 2019 04:00:09 GMT https://community.splunk.com/t5/Splunk-Search/Lookups-slow-performance/m-p/397845#M115449 cwinkler109 2019-02-26T04:00:09Z https://community.splunk.com/t5/Splunk-Search/Lookups-slow-performance/m-p/397846#M115450 <P>The second search is going directly for the index to look for your guid. The first one needs to match (don't know how many fields are you matching against the lookup) every event to the lookup before filtering, and that's where this questions come into place</P> <P>Is that lookup CSV or KVStore based?</P> <P>If it is KVStore, maybe you'd prefer it to be replicated to the Indexer layer for performance increase?</P> <P>How frequently is that lookup updated?</P> Tue, 26 Feb 2019 13:51:55 GMT https://community.splunk.com/t5/Splunk-Search/Lookups-slow-performance/m-p/397846#M115450 tiagofbmm 2019-02-26T13:51:55Z https://community.splunk.com/t5/Splunk-Search/Lookups-slow-performance/m-p/397847#M115451 <P>It depends on how many definitions exist to create the field <CODE>guid</CODE> and what types they are. This is a VERY deep topic. Start here and pay special attention to the parts regarding <CODE>lispy</CODE>:<BR /> <A href="https://conf.splunk.com/files/2017/slides/fields-indexed-tokens-and-you.pdf">https://conf.splunk.com/files/2017/slides/fields-indexed-tokens-and-you.pdf</A><BR /> Also check out this:<BR /> <A href="https://splunkbase.splunk.com/app/2871/">https://splunkbase.splunk.com/app/2871/</A></P> <P>If the field <CODE>guid</CODE> is an indexed field, then you can use this:</P> <PRE><CODE>index=my_index guid::my_guid </CODE></PRE> Sun, 03 Mar 2019 05:14:09 GMT https://community.splunk.com/t5/Splunk-Search/Lookups-slow-performance/m-p/397847#M115451 woodcock 2019-03-03T05:14:09Z