Splunk Search

HiddenPostProcess clarification

aweitzman
Motivator

The limitations for HiddenPostProcess are described as follows:

"If the base search is a non-transforming search, Splunk retains only the first 10,000 events returned."

What if the search is a transforming search, but it returns a data cube with more than 10,000 entries? So, say, 30 days' worth of one-day counts of some information about 500 different users? Does any information get thrown out when that happens?

Thank you.

Tags (1)
0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

It should still truncate records to its default limit 10,000. Possible workaround is to add following parameters to HiddenPostProcess.

<param name="maxCount">100000</param>

Updates

If you're using HiddenSearch Module (with HiddenPostProcess module), add following to HiddenSearch module to increase the limit of HiddenPostProcess module (default to 10,000)

   <param name="maxCount">100000</param>

If you're using HiddenSavedSearch module with HiddenPostProcess module), then replace HiddenSavedSearch module with HiddenSearch module, updated search string as mentioned below and added maxCount parameter to increase the limit of HiddenPostProcess module (default to 10,000)

Replace

<module name="HiddenSavedSearch" .....>
    <param name="savedSearch">yoursearchname</param>
    ....
      <module name="HiddenPostProcess" ...>
      ....

with

<module name="HiddenSearch" ...> 
    <param name="search">|savedsearch yoursearchname</param> 
    <param name="maxCount">100000</param>
    ....
      <module name="HiddenPostProcess" ...>
    ....

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

It should still truncate records to its default limit 10,000. Possible workaround is to add following parameters to HiddenPostProcess.

<param name="maxCount">100000</param>

Updates

If you're using HiddenSearch Module (with HiddenPostProcess module), add following to HiddenSearch module to increase the limit of HiddenPostProcess module (default to 10,000)

   <param name="maxCount">100000</param>

If you're using HiddenSavedSearch module with HiddenPostProcess module), then replace HiddenSavedSearch module with HiddenSearch module, updated search string as mentioned below and added maxCount parameter to increase the limit of HiddenPostProcess module (default to 10,000)

Replace

<module name="HiddenSavedSearch" .....>
    <param name="savedSearch">yoursearchname</param>
    ....
      <module name="HiddenPostProcess" ...>
      ....

with

<module name="HiddenSearch" ...> 
    <param name="search">|savedsearch yoursearchname</param> 
    <param name="maxCount">100000</param>
    ....
      <module name="HiddenPostProcess" ...>
    ....

aweitzman
Motivator

This seems to work. If you post another answer that combines the information from all these comments into one post, I will mark it as the answer.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

It seems that options is not available with HiddenSavedSearch module. Workaround will be to use HiddenSearch module and in the 'search' param use "|savedsearch YourSavedSearchName"

aweitzman
Motivator

This doesn't seem to work if I'm starting with a HiddenSavedSearch module. Is that not allowed?

0 Karma

somesoni2
SplunkTrust
SplunkTrust

To the main search (HiddenSearch module).

0 Karma

aweitzman
Motivator

Do I add this to the main search, or to each postprocess search?

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...