I'm watching the Fundamentals 2 course (finally XD) and I've come across the search ending with something like: | sort -field | rename field2 as something_else | fields - field3 And the question is whether it would be a bit faster to first remove the field and then sort? Or is it the other way around? On the one hand - removing fields should give you less data to manipulate when sorting. On the other hand - I don't expect Splunk to physically rewrite each and every event on each pipe so it might not really matter at all. Side question - let's assume we rewrite it into | search field2=something | fields - field3 In this case - is it better to first trim the event set and then remove field or first remove field and then trim? Of course I know that probably it's completely insignificant compared to the time it takes to get the data from the indexes. But that's just me digging into the internals 😉 ... View more

Following https://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Sharedatamodelsummaries I set up sharing acceleration summaries between two search-head clusters. I found guid of one of the clusters, set it up as a source_guid into a default stanza on the other cluster (first cluster uses CIM app and ES, the second one has just CIM app with datamodel settings migrated from first cluster). So datamodel settings on the second cluster is  a subset of settings from the first cluster (I did a btool dump of dataset settings and compared them with vimdiff). On first cluster I have some addiional datamodels from ES app, the rest datasets is identical on both clusters (of course apart from the source_guid attribute). As far as I understand the article, it should just work. But as far as I add the CIM app (define the datamodels) on the second cluster, it starts killing my indexers. I have 20CPU nodes with 64G of RAM and their load is typicaly around 6-7 and memory usage doesn't exceed 40G. Since the added the CIM app, load is doesn't fall below 40(!) and sometimes jumps to around 45 and the RAM is all used (I  even get oom-killers every half an hour or so). The monitoring console shows that most resources (by a great margin) is used by datamodel acceleration. And the top memory-consuming searches are various instances of _ACCELERATE_DM_Splunk_SA_CIM_Network_Traffic_ACCELERATE_ I don't understand however: 1) Why doesn't splunk just use the data I pointed it to? It seems to be "rebuilding" the summaries (and yes, I have a lot of network data, so it makes sense) 2) Why does it spawn the consecutive acceleration searches when the old ones didn't complete yet? ... View more

I'm reading the docs about sharing summaries between search-heads and I'm a bit puzzled. https://docs.splunk.com/Documentation/Splunk/8.2.1/Knowledge/Sharedatamodelsummaries The article states: "You can find the GUID for a search head cluster in the [shclustering] stanza of server.conf. If you are running a single instance you can find the GUID in etc/instance.cfg. "  But in my case the only guids I can find are those on single shcluster members in etc/instance.cfg. Of course each one is different. I cannot seem to find a "search head cluster GUID" anywhere. What am I doing wrong? ... View more