HiddenPostProcess clarification

[aweitzman]

The limitations for HiddenPostProcess are described as follows:

"If the base search is a non-transforming search, Splunk retains only the first 10,000 events returned."

What if the search is a transforming search, but it returns a data cube with more than 10,000 entries? So, say, 30 days' worth of one-day counts of some information about 500 different users? Does any information get thrown out when that happens?

Thank you.

[somesoni2 · Accepted solution]

It should still truncate records to its default limit 10,000. Possible workaround is to add following parameters to HiddenPostProcess.

<param name="maxCount">100000</param>

---

Updates

If you're using HiddenSearch Module (with HiddenPostProcess module), add following to HiddenSearch module to increase the limit of HiddenPostProcess module (default to 10,000)

   <param name="maxCount">100000</param>

If you're using HiddenSavedSearch module with HiddenPostProcess module), then replace HiddenSavedSearch module with HiddenSearch module, updated search string as mentioned below and added maxCount parameter to increase the limit of HiddenPostProcess module (default to 10,000)

Replace

<module name="HiddenSavedSearch" .....>
    <param name="savedSearch">yoursearchname</param>
    ....
      <module name="HiddenPostProcess" ...>
      ....

with

<module name="HiddenSearch" ...> 
    <param name="search">|savedsearch yoursearchname</param> 
    <param name="maxCount">100000</param>
    ....
      <module name="HiddenPostProcess" ...>
    ....

[aweitzman]

This seems to work. If you post another answer that combines the information from all these comments into one post, I will mark it as the answer.

[somesoni2]

It seems that options is not available with HiddenSavedSearch module. Workaround will be to use HiddenSearch module and in the 'search' param use "|savedsearch YourSavedSearchName"

[aweitzman]

This doesn't seem to work if I'm starting with a HiddenSavedSearch module. Is that not allowed?

[somesoni2]

To the main search (HiddenSearch module).

[aweitzman]

Do I add this to the main search, or to each postprocess search?