Hi I am trying to automate alert set up for splunk alerts . I am using splunk tf provider : https://registry.terraform.io/providers/splunk/splunk/latest/docs/resources/saved_searches#argument-reference   We have couple of actions  when an alert is generated like email , slack message & call out . Call out is using custom action called xmatter. Curl command creates alert well curl -ks -u username:password https://<splunkurl>:8089/servicesNS/nobody/digital_dcps_sre_search/saved/searches -d name=No_Memory_Left -d cron_schedule="*/5 * * * *" -d description="This test job is a durable saved search" -d dispatch.earliest_time="-24h@h" -d dispatch.latest_time="now" -d action.digital_slack="1" -d action.digital_slack.param.channel="#dcps-sre-alerts" -d action.digital_slack.param.message="Kong DP Alert .This line indicates that the data plane instance is trying to read a config from the control plane that is bigger than the config cache shared memory location. This means the data plane can no longer receive configuration updates." -d action.digital_slack.param.workspace="rbwm" -d alert.track="true" -d alert_comparator="greater than" -d alert_threshold="1" -d is_scheduled="true" -d alert_type="number of events" -d action.abc_xmatters_alerts="0" -d action.abc_xmatters_alerts.param.key="xxxxxx" -d action.abc_xmatters_alerts.param.severity="MINOR" -d action.abc_xmatters_alerts.param.summary="text=UK TP Splunk Alert. '$name$' alert was triggered. $result.final_gateway_url$. Link: $results_link$\napplication=Technical-Platform-Engineering_UK" -d actions="digital_slack" -d alert.digest_mode="true" -d alert.expires="5d" -d alert.severity="4" -d alert.suppress="true" -d alert.suppress.period="60m" -d description="This line indicates that the data plane instance is trying to read a config from the control plane that is bigger than the config cache shared memory location. This means the data plane can no longer receive configuration updates" -d disabled="false" --data-urlencode search="search index=digital_technical_onprem_kongdp_raw sourcetype=SystemErr \\[clustering\\] unable to update running config: no memory"   But it fails if I try to use splunk tf provider because action_param_key etc are not supported . Is there anyway I can set customer action in alert using tf ?   resource "splunk_saved_searches" "No_Memory_Left" { cron_schedule = "*/5 * * * *" dispatch_earliest_time = "-24h@h" dispatch_latest_time = "now" #action_slack = "1" action_slack_param_channel = "#dcps-sre-alerts" action_slack_param_message = "Kong DP Alert .This line indicates that the data plane instance is trying to read a config from the control plane that is bigger than the config cache shared memory location. This means the data plane can no longer receive configuration updates. Refer " #action_slack_param_workspace = "rbwm" alert_track = "true" alert_comparator = "greater than" alert_threshold = "1" is_scheduled = "true" alert_type = "number of events" #action_abc_xmatters_alerts = "0" #action_param_key = "xxxxxx" #action_param_severity = "MINOR" #action_param_summary = "text=UK TP Splunk Alert. '$name$' alert was triggered. $result.final_gateway_url$. Link: $results_link$\napplication=Technical-Platform-Engineering_UK" actions = "digital_slack" alert_digest_mode = "true" alert_expires = "5d" alert_severity = "4" alert_suppress = "true" alert_suppress_period = "60m" description = "This line indicates that the data plane instance is trying to read a config from the control plane that is bigger than the config cache shared memory location. This means the data plane can no longer receive configuration updates" disabled = "false" search = "search index=digital_raw sourcetype=SystemErr \\[clustering\\] unable to update running config: no memory" name = "No_Memory_Left" acl { app = "digital_dcps_sre_search" owner = "GB-SVC-DSRE-SPL" sharing = "app" } } ... View more

Hi. I am running below search. Sometimes error does not happen but in that case, stats command shows no data. Can I show column still with some default value when the error is not happening? I want to show no data as 'NOERRORHere' ? index=*tech* index=digital_technical_api_gw_hac_raw " New Message Response notification related " | join type=outer host [|inputlookup gw_list] | stats values(apiName) AS API ,values(host) AS host count(eval(statusCode>400)) AS HttpResponseMoreThan400, count(eval(statusCode<400)) AS HttpResponseLessThan400 tp_set , source | join type=outer max=0 [ search index=*tech* index=digital_technical_api_gw_hac_raw ("com.ncipher.nfast.connect.StatusNotOK" OR "An Exception was caught while trying to decrypt the data") | join type=outer host [|inputlookup gw_list] | stats count as HSMErrorCountsOnGivenStripe by tp_set , source ] ... View more

Hi I have two searches search a : index=*tech* sourcetype=technical_rproxy_access OR sourcetype=technical_mule_api NOT statusCode="0" | rex field=source "/appvol/(?[\w\/]+)/logs.*" | stats count as GWcount values(apiName) as APIatGW ,values(statusCode) as StatusatGW by MULE search b: index=* sourcetype=technical_rproxy_access | rex field=api_name "api\/(?[\w\-]+)*" |rename api_name as apiName|stats count as RPCount values(apiName) as APIatRproxy , values(status) as StatusatRproxy I have combined two as below : index=*tech* sourcetype=technical_rproxy_access OR sourcetype=technical_mule_api NOT statusCode="0" | rex field=source "/appvol/(?[\w\/]+)/logs.*" | stats count as GWcount values(apiName) as APIatGW ,values(statusCode) as StatusatGW by MULE | join type=left max=0 apiName [ search index=* sourcetype=technical_rproxy_access | rex field=api_name "api\/(?[\w\-]+)*" |rename api_name as apiName|stats count as RPCount values(apiName) as APIatRproxy , values(status) as StatusatRproxy] I want to join two results such that apiName will be common in both, but my result is not working. ... View more