Splunk Search

Splunk problem with fast and smart mode

virtuosoo
Explorer

Hello community,

I am facing a problem ,I have an instance of splunk installed on linux server , And I am trying to do a copy of this instance on my localhost which turns on windows machine.So I have done copy of all my apps and indexes.But I find out that while doing my requests, search commands like stats, timechart ....are working only in verbose mode, And they are returning no data in fast and smart mode.

Any help please,

N.B. : -Version splunk on linux : 7.2.0 -Version splunk on my local machine : 7.2.3

alt text

alt text

0 Karma

ecoquelin
Explorer

I have the same issue as @virtuosoo and I confirm the issue seems not directly related to the SMART or VERBOSE mode.

Let's take a more classic case:

index=_audit | stats count

We will all agree that it should work even in SMART or FAST mode. Because as per the documentation, field discovery doesn't occur (as per @woodcock message) but metafields are always "discovered" (such as sourcetype, source, etc.). As a result, we should expect a result to the command above.

When this command is executed on a Splunk running on Windows, we have 0 results. Same command on a Splunk running on Linux gives the expected number. The same command in VERBOSE mode gives the expected result on both environments.

If I update the command as follows:

index=_audit | head 1000 | stats count

Even in SMART/VERBOSE mode on Windows, I get "1000". Here, I assume that the "head" command forces the field discovery in a way...

How can we explain that the behavior differs from a Splunk running on Windows compared to a Splunk running on Linux ?

Please help.

0 Karma

ecoquelin
Explorer

Still looking for an answer as this problem can not be explained.

Going through the logs, we noticed that between Linux and Windows, the search log differs:

On Linux, I have
03-21-2019 18:43:56.676 INFO CMBucketId - CMIndexId: New indexName=audit inserted, mapping to id=2
03-21-2019 18:43:56.676 INFO BatchSearch - index: _audit dbsize=1
03-21-2019 18:43:56.676 INFO UnifiedSearch - Initialization of search data structures took 6 ms
03-21-2019 18:43:56.676 INFO UnifiedSearch - Processed search targeting arguments
03-21-2019 18:43:56.676 INFO StatsProcessor - No group-by fields specified, incompatible for high cardinality improvements
03-21-2019 18:43:56.676 INFO LocalCollector - Final required fields list = _subsecond,index,prestats_reserved
,psrsvd_
03-21-2019 18:43:56.676 INFO UserManager - Unwound user context: ecoquelin -> NULL
03-21-2019 18:43:56.676 INFO UserManager - Setting user context: ecoquelin
03-21-2019 18:43:56.676 INFO UserManager - Done setting user context: NULL -> ecoquelin
03-21-2019 18:43:56.676 INFO BatchSearch - Searching index:_audit with LISPY:'[ AND ]'
03-21-2019 18:43:56.686 INFO SearchOperator:kv - no fields required. Running only non-optimizeable extractions ...
03-21-2019 18:43:56.700 INFO UserManager - Unwound user context: ecoquelin -> NULL

On Windows, I have
03-21-2019 18:39:28.928 INFO CMBucketId - CMIndexId: New indexName=audit inserted, mapping to id=1
03-21-2019 18:39:28.930 INFO BatchSearch - index: _audit dbsize=5
03-21-2019 18:39:28.930 INFO UnifiedSearch - Initialization of search data structures took 7 ms
03-21-2019 18:39:28.930 INFO UnifiedSearch - Processed search targeting arguments
03-21-2019 18:39:28.930 INFO StatsProcessor - No group-by fields specified, incompatible for high cardinality improvements
03-21-2019 18:39:28.930 INFO LocalCollector - Final required fields list = _subsecond,index,prestats_reserved
,psrsvd_
03-21-2019 18:39:28.930 INFO UserManager - Unwound user context: reda -> NULL
03-21-2019 18:39:28.930 INFO UserManager - Setting user context: reda
03-21-2019 18:39:28.930 INFO UserManager - Done setting user context: NULL -> reda
03-21-2019 18:39:28.931 INFO BatchSearch - Searching index:_audit with LISPY:'[ AND ]'
03-21-2019 18:39:28.932 ERROR LocalCollector - sid: bad allocation

Field extraction algorithm may be different from Windows to Linux...

Any pointer ?

Please help

0 Karma

woodcock
Esteemed Legend

That is how fast and smart mode work! They are faster because either they create NO extra fields ( fast ) or only the ones that you explicitly use in your search ( smart ). In your case, you only reference count and your by field, so that is all that is available on the Events tab.

0 Karma

jgallo92
Engager

Hi!
Try to add | fields * | before your stats.

0 Karma

pkeenan87
Communicator

Are you on the events tab? Commands like stats and timechart will return statistics and events in Verbose Mode but only statistics in fast and smart mode.

0 Karma

virtuosoo
Explorer

Thank you for your reply,

I have updated the issue by uploading Images to show the problem.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...