Splunk Enterprise Security

SavedSearch running as type=Inline works, type=Saved fails - why?


I setup a saved search and it is failing to run. It is throwing an error in the gui

Error in 'sendalert' command: Alert script returned error code 3.

but I happened to create another when trying to debug it and that one worked. What I can see different is the the one that works has these two key lines in search.log

SavedSplunk - Savedsearch scheduling at the 'application' level is only effective the for 'nobody' user. Disabling schedule of savedsearch_ident="admin;SplunkEnterpriseSecuritySuite;Cancellations"

followed by

sendmodalert - Invoking modular alert action=risk for search="Cancellations" sid="scheduler__admin__SplunkEnterpriseSecuritySuite__Cancellations_at_1569907560_121" in app="SplunkEnterpriseSecuritySuite" owner="admin" type="**inline**"

whereas the failing one does not have the first line, but has this for the second

sendmodalert - Invoking modular alert action=risk for search="Cancellations" sid="scheduler__admin__SplunkEnterpriseSecuritySuite__Cancellations_at_1569910380_349" in app="SplunkEnterpriseSecuritySuite" owner="admin" type="**saved**"

key difference being type=inline vs saved

Just wondering what that first line means and if there is a way to always force a saved search to run inline in all cases

0 Karma


The background to this is that I am running Enterprise Security. I was hoping to assign a risk score to multiple objects, but a correlation search cannot run more than one adaptive response action for risk.

So, I am implementing a saved search instead that will

  • create a score/object/type tuple for each search result
  • mvexpand on this field
  • Split out the field
  • Run "sendalert risk" for each of the resulting events

Appendpipe does not solve the problem for more than two risk objects, as you end up with 2^(n-1) events where n is the number of risk objects.

The saved search works when run manually, but fails when scheduled.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!