I have a questions about custom search commands and the streaming_preop option. Is there some reason why the preopt is only honored if retevs (retainsevents) is false?
I have a situation where I would like to run a pre-processing command, and I want my search script to return events not results. As soon as I set retevs=True, then the pre-operation search command is not executed. There are other limitations on the streaming_preop listed in the docs, but there is nothing mentioned about any conflicts with retainsevents.
Just to be clear, this results in "addinfo" being called:
We don't run the streaming_preop if your command isn't the first reporting command. So basically you need to be a reporting command (retainsevents=false), and also you have to the first one. This is so that a reporting command can specify a optimization that will reduce what comes back from the indexers to only the sufficient statistics needed by that reporting command.
You can specify that your pre-op is required via the requires_preop setting, but that only defeats the second requirement. There is no way that you can force a preop to be run if your command is not a reporting command.