Splunk Search

metadata command giving different answers on different search heads

SplunkTrust
SplunkTrust

I recently loaded a 4.2 search head onto my laptop in order to use it for testing some view development in a way that I wouldn't impact production users but could still use production event data. On my new search head, I've noticed the dashboard_live screen in the search app is off by several billion events compared to our production search head. Running this search on the two search heads:

| metadata type=sourcetypes

will give back different results. The one on my laptop excludes one of my sourcetypes with over 5 billion events. The one on the main search head/indexer (which is 64 bit btw) does not. Given my laptop is a 32-bit machine, is just just a case of the count is too big for a 32-bit integers or is something else afoot?

Tags (1)
1 Solution

Splunk Employee
Splunk Employee

I think you got it right and the count returned by the metadata command is limited by your 32bit system to approximately 2 billion events.

If you want to perform further testing to confirm this, you could scope metadata to some individual indexes with the "index=" argument and compare those values. Try to get close to 2 billion events and see if that's the breaking point.

You can also try the following and see if it breaks 😄

* | head 1 | eval test=2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2+1 | eval result=if(test>2147483647,"win","lose!") | table test, result

Let us know, I'm curious but too lazy to find a 32 bit instance right now.

View solution in original post

Splunk Employee
Splunk Employee

I think you got it right and the count returned by the metadata command is limited by your 32bit system to approximately 2 billion events.

If you want to perform further testing to confirm this, you could scope metadata to some individual indexes with the "index=" argument and compare those values. Try to get close to 2 billion events and see if that's the breaking point.

You can also try the following and see if it breaks 😄

* | head 1 | eval test=2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2*2+1 | eval result=if(test>2147483647,"win","lose!") | table test, result

Let us know, I'm curious but too lazy to find a 32 bit instance right now.

View solution in original post

SplunkTrust
SplunkTrust

Well, the eval works well above 32 bits. But, metadata limited to a specific index leaves out a sourcetype with > 4B events in it. This breakage may be specific to metadata... But it's not a big deal, it's just a test search head.

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!