Knowledge Management

How to search that only the scheduled search adds results to the summary index

MonkeyK
Builder

Slightly indirect question.

What I am really trying to do is to ensure that only the scheduled search adds results to the summary index.

For lookup tables, the report scheduling dialog box lets me specify that search results should go to a lookup table. This is nice because if I edit the search, I don't have to worry about accidentally updating the lookup table.

Unfortunately, report scheduling does not have a similar option for summary indices (#featurerequest), so I was thinking that I could roll my own if the report could know that it is running as part of a schedule. Something like

<base search>
| eval isScheduled=check_scheduling_info()
| search isScheduled
|<summarize data>
|collect index=my_summary marker="search_name=my_search"

or maybe somehow conditionally write to the test index instead of my_summary if the search is not running as part of a schedule.

Is this something that can be done? Is there a better way to go about what I really want?

0 Karma
1 Solution

woodcock
Esteemed Legend

Here is a macro definition from my macros.conf that does this:

[SIEMMacro_Collect_Conditional(2)]
args = stringGeneratingSearchName_INPUT, summaryIndexName_INPUT
definition = addinfo\
\
| rename COMMENT1of2 AS "If the scheduler is running this, it WILL get written to the SI (2nd stanza) but"\
| rename COMMENT2of2 AS "If a person is running ad-hoc, it will NOT get written to the SI (1st stanza)."\
| multireport\
\
[ eval AdHocUserRun=if(match(info_sid, "^scheduler__"), "N", "SoNotSavedToSI")\
| search NOT AdHocUserRun="N" ]\
\
[ eval AdHocUserRun=if(match(info_sid, "^scheduler__"), "N", "Y")\
| search AdHocUserRun="N" | fields - AdHocUserRun\
| collect $summaryIndexName_INPUT$ addtime=t marker="report_name=\"$stringGeneratingSearchName_INPUT$\"" ]
errormsg = Description: Author = Gregg Woodcock
iseval = 0

View solution in original post

woodcock
Esteemed Legend

You can remove the | collect from the SPL of your saved search and use the alert/report action to output results to lookup and than have another search that runs every few minutes to copy data from the lookup to the summary index. There is no reason for anybody to run the 2nd search because it is a system-level housekeeping search. But the main search which will be run ad-hoc for many good reasons is now safe to run by people at any time.

0 Karma

woodcock
Esteemed Legend

Here is a macro definition from my macros.conf that does this:

[SIEMMacro_Collect_Conditional(2)]
args = stringGeneratingSearchName_INPUT, summaryIndexName_INPUT
definition = addinfo\
\
| rename COMMENT1of2 AS "If the scheduler is running this, it WILL get written to the SI (2nd stanza) but"\
| rename COMMENT2of2 AS "If a person is running ad-hoc, it will NOT get written to the SI (1st stanza)."\
| multireport\
\
[ eval AdHocUserRun=if(match(info_sid, "^scheduler__"), "N", "SoNotSavedToSI")\
| search NOT AdHocUserRun="N" ]\
\
[ eval AdHocUserRun=if(match(info_sid, "^scheduler__"), "N", "Y")\
| search AdHocUserRun="N" | fields - AdHocUserRun\
| collect $summaryIndexName_INPUT$ addtime=t marker="report_name=\"$stringGeneratingSearchName_INPUT$\"" ]
errormsg = Description: Author = Gregg Woodcock
iseval = 0

MonkeyK
Builder

Super useful! Thank you!

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...