topic Re: HiddenPostProcess clarification in Splunk Search

HiddenPostProcess clarification

aweitzman — Fri, 11 Apr 2014 14:23:42 GMT

The limitations for HiddenPostProcess are described as follows:

"If the base search is a non-transforming search, Splunk retains only the first 10,000 events returned."

What if the search is a transforming search, but it returns a data cube with more than 10,000 entries? So, say, 30 days' worth of one-day counts of some information about 500 different users? Does any information get thrown out when that happens?

Thank you.

Re: HiddenPostProcess clarification

somesoni2 — Fri, 11 Apr 2014 14:45:05 GMT

It should still truncate records to its default limit 10,000. Possible workaround is to add following parameters to HiddenPostProcess.

<param name="maxCount">100000</param>

Updates

If you're using HiddenSearch Module (with HiddenPostProcess module), add following to HiddenSearch module to increase the limit of HiddenPostProcess module (default to 10,000)

   <param name="maxCount">100000</param>

If you're using HiddenSavedSearch module with HiddenPostProcess module), then replace HiddenSavedSearch module with HiddenSearch module, updated search string as mentioned below and added maxCount parameter to increase the limit of HiddenPostProcess module (default to 10,000)

Replace

<module name="HiddenSavedSearch" .....>
    <param name="savedSearch">yoursearchname</param>
    ....
      <module name="HiddenPostProcess" ...>
      ....

with

<module name="HiddenSearch" ...> 
    <param name="search">|savedsearch yoursearchname</param> 
    <param name="maxCount">100000</param>
    ....
      <module name="HiddenPostProcess" ...>
    ....

Re: HiddenPostProcess clarification

aweitzman — Fri, 11 Apr 2014 17:31:54 GMT

Do I add this to the main search, or to each postprocess search?

Re: HiddenPostProcess clarification

somesoni2 — Fri, 11 Apr 2014 17:57:23 GMT

To the main search (HiddenSearch module).

Re: HiddenPostProcess clarification

aweitzman — Fri, 11 Apr 2014 20:11:05 GMT

This doesn't seem to work if I'm starting with a HiddenSavedSearch module. Is that not allowed?

Re: HiddenPostProcess clarification

somesoni2 — Fri, 11 Apr 2014 20:49:40 GMT

It seems that options is not available with HiddenSavedSearch module. Workaround will be to use HiddenSearch module and in the 'search' param use "|savedsearch YourSavedSearchName"

Re: HiddenPostProcess clarification

aweitzman — Mon, 28 Apr 2014 14:08:43 GMT

This seems to work. If you post another answer that combines the information from all these comments into one post, I will mark it as the answer.