Custom search command: preop only works when retainevents is false?

[Lowell]

I have a questions about custom search commands and the streaming_preop option. Is there some reason why the preopt is only honored if retevs (retainsevents) is false?

I have a situation where I would like to run a pre-processing command, and I want my search script to return events not results. As soon as I set retevs=True, then the pre-operation search command is not executed. There are other limitations on the streaming_preop listed in the docs, but there is nothing mentioned about any conflicts with retainsevents.

Just to be clear, this results in "addinfo" being called:

 # streaming, generating, retevs, reqsop, preop
 splunk.Intersplunk.outputInfo(False, False, False, True, "addinfo")

But, in this case "addinfo" is NOT called before my search command:

 # streaming, generating, retevs, reqsop, preop
 splunk.Intersplunk.outputInfo(False, False, True, True, "addinfo")

Any ideas?

[steveyz]

We don't run the streaming_preop if your command isn't the first reporting command. So basically you need to be a reporting command (retainsevents=false), and also you have to the first one. This is so that a reporting command can specify a optimization that will reduce what comes back from the indexers to only the sufficient statistics needed by that reporting command.

You can specify that your pre-op is required via the requires_preop setting, but that only defeats the second requirement. There is no way that you can force a preop to be run if your command is not a reporting command.