https://community.splunk.com/t5/Splunk-Search/Why-kv-store-lookup-is-much-slower-than-csv-lookup/m-p/484544#M135635 <P>Hi everyone!<BR /> We've moved some of heavy lookups to kv store and now they work faster and more stable. But one of them barely can calculate something, being inserted in a search with <CODE>lookup</CODE> even on 24 hour span, while it easily loads with <CODE>inputlookup</CODE>.</P> <P>Accelerated field is a hexadecimal uid of 16 symbols. To make it easier for splunk to accelerate it, we converted it in a bigint and accelerated on this new field. Still the same slowness. It was rewritten but it didn't help. And its csv version lookups much faster, but we had some issues with updating large lookups so we moved to kv.</P> <P>Slow-lookuped kv has 10mln entries and around 5 fields.<BR /> Fast-lookuped kv with 40mln entries and 10 fields works fine (with accelerated field also converted from hex to bigint)</P> <P>Server: 251.81 GB Physical Memory, 20 CPU Cores</P> <P>in limits.conf:<BR /> max_threads_per_outputlookup = 10</P> <P>in server.conf:<BR /> oplogSize = 2000</P> <P>What can cause such slowness? What params can we tune?</P> Wed, 30 Sep 2020 05:10:19 GMT iKate 2020-09-30T05:10:19Z https://community.splunk.com/t5/Splunk-Search/Why-kv-store-lookup-is-much-slower-than-csv-lookup/m-p/484544#M135635 <P>Hi everyone!<BR /> We've moved some of heavy lookups to kv store and now they work faster and more stable. But one of them barely can calculate something, being inserted in a search with <CODE>lookup</CODE> even on 24 hour span, while it easily loads with <CODE>inputlookup</CODE>.</P> <P>Accelerated field is a hexadecimal uid of 16 symbols. To make it easier for splunk to accelerate it, we converted it in a bigint and accelerated on this new field. Still the same slowness. It was rewritten but it didn't help. And its csv version lookups much faster, but we had some issues with updating large lookups so we moved to kv.</P> <P>Slow-lookuped kv has 10mln entries and around 5 fields.<BR /> Fast-lookuped kv with 40mln entries and 10 fields works fine (with accelerated field also converted from hex to bigint)</P> <P>Server: 251.81 GB Physical Memory, 20 CPU Cores</P> <P>in limits.conf:<BR /> max_threads_per_outputlookup = 10</P> <P>in server.conf:<BR /> oplogSize = 2000</P> <P>What can cause such slowness? What params can we tune?</P> Wed, 30 Sep 2020 05:10:19 GMT https://community.splunk.com/t5/Splunk-Search/Why-kv-store-lookup-is-much-slower-than-csv-lookup/m-p/484544#M135635 iKate 2020-09-30T05:10:19Z