if it helped, I would really appreciate if you could pick my response as solution.  Happy Splunking 🙂  ... View more

Regardless of it's severity , it would stay in the list as long as retention time of the it's kvstore: itsi_grouped_alerts (setting is TTL) This could be useful when episode is closed, and one wants check what was the resolution notes from prev. incident etc.. That's being said, if you must delete episodes after certain time: find the collections.conf where itsi_grouped_alerts is configured ( I think it was in main ITSI app but could be in one of the side apps) [itsi_grouped_alerts] ttl = 2592000 # 30 days in seconds if you don't wanna deal with conf files but with pure SPL, you can schedule a daily scheduled search that will do the same work manually for you: | inputlookup itsi_grouped_alerts | where _time < relative_time(now(), "-30d") | outputlookup itsi_grouped_alerts append=false  please note this is un-reversible.  Finally, setting a retention time on indexes.conf for notable index would do the final work for episodes.  ... View more

1) I think you should see itsi indexes :  itsi_tracked_alerts and itsi_grouped_alerts , locate this alert and go from there. there should be a service ID in the event that you can trace to update on the way.  2) only think I can think of is private view , perhaps you should share more context. ... View more

How about using multi-KPI alert using the service score?   ... View more

I would recommend trying : 1) Since you said size is small, try untarring (unzipping) it to a log file then try again. 2) if above doesn't work, try putting file into filesystem and input with continuous monitoring using its native path.    ... View more

is it possible that these lookups saved in private? meaning not shared globally or with app level Settings > Lookups > Lookup table files Then find your lookup file , check under Sharing column  ... View more

sorry for the double dashes, it should be single dash as your previously successful apply command: splunk apply shcluster-bundle -action stage --answer-yes -auth admin:'password' -preserve-lookups false splunk apply shcluster-bundle -action send -target https://172.26.35.95:8089 --answer-yes -auth admin:'password' -preserve-lookups false Please note that this is going to override all lookups, if you have some lookup that you want to retain, move it out of the shcluster-apps directory.  Please note that I don't use -action in my bundle push. but this should still work ... View more

Splunk's Best practice advises using Search Head Deployer to push applications to Search Head Cluster, However, you can still use deployment server to push your new app to Search Head Deployer , then to be pushed to SHC.  This is important in your case, since SHD has a flag in its CLI command to preserve lookups when pushing new configuration to Search Head Cluster Members. If you must use your Deployment server, I can advise on configuration on the DS to push your new app /shcluster-apps in the Search Head Deployer. However, starting from SHD: you would need below command to override CSVs in the Search Head: splunk apply shcluster-bundle --target <SHC_member_URI> --preserve-lookups false   ... View more

is this a search head cluster or single search heads ? ... View more

I don't have a stream app and my lab is not suitable for this at this time. However, I believe we can find the solution after a btool run: Could you run below on the CLI and find where it registers selected index in your server (I would expect there is an indexes.conf file bundled with the app, and inputs.conf file pointing to that index): Test below after picking an index from dropdown and grep below for this selected index:   #Find which indexes.conf for the picked index is stored $SPLUNK_HOME$/bin/splunk btool indexes list --debug #Find where is the inputs.conf pointing for the same $SPLUNK_HOME$/bin/splunk btool inputs list --debug     This should point you to right place ... View more

I believe there is no direct way of doing this unless you hit to ServiceNow directly via REST with custom alert action or similar. However, this may work for the purpose of displaying it in the incident: ... your | makeresults .... | eval html_row="<tr><td>" . Date . "</td><td>" . Branch . "</td><td>" . FileName . "</td><td>" . ErrorReason . "</td></tr>" | stats list(html_row) as rows | eval html_table="<table border='1'><tr><th>Date</th><th>Branch</th><th>FileName</th><th>ErrorReason</th></tr>" . mvjoin(rows, "") . "</table>" then using $result.html_table$ in the description field. if ServiceNow doesn't parse html, then in a similar fashion, concatenating result set into one field with some formatting should do the same. .... your |makeresults ... | eval header="Date | Branch | FileName | ErrorReason #NEXT#" | eval myfield=Date." | ".Branch." | ".FileName." | ".ErrorReason."#NEXT#" | eventstats list(myfield) as mylist | nomv mylist | eval resultset=header.mylist | rex mode=sed field=resultset "s/#NEXT#/\n/g" | fields resultset then using $result.resultset$ in the description field. this is not the exact solution but could be a good work around.  ... View more

if this work-around worked, could you pick it as an answer ? ... View more

Are you looking for general metrics/usage or metrics per search/alert ?  ... View more

I believe there is no direct way of doing this unless you hit to Servicenow directly via REST with custom alert action or similar. However, this may work for the purpose of displaying it in the incident: ... your | makeresults .... | eval html_row="<tr><td>" . Date . "</td><td>" . Branch . "</td><td>" . FileName . "</td><td>" . ErrorReason . "</td></tr>" | stats list(html_row) as rows | eval html_table="<table border='1'><tr><th>Date</th><th>Branch</th><th>FileName</th><th>ErrorReason</th></tr>" . mvjoin(rows, "") . "</table>" then using $result.html_table$ in the description field. if ServiceNow doesn't parse html, then in a similar fashion, concatenating result set into one field with some formatting should do the same. .... your |makeresults ... | eval header="Date | Branch | FileName | ErrorReason #NEXT#" | eval myfield=Date." | ".Branch." | ".FileName." | ".ErrorReason."#NEXT#" | eventstats list(myfield) as mylist | nomv mylist | eval resultset=header.mylist | rex mode=sed field=resultset "s/#NEXT#/\n/g" | fields resultset then using $result.resultset$ in the description field. this is not the exact solution but could be a work around.  ... View more

Could you share your alert search and values you pass on to ServiceNow incident integration ?  ... View more

according to screenshot, it queried DNS server and returned cloud native names. From the table fields, IP is queried against and hostname returned. cmdb_name is just a new field in case DNS lookup doesn't return anything.  ... View more

I would first regex out then do DNS lookup to a separate field followed by coalesce in case DNS server doesn't know the host name. Here is the SPL with make results: | makeresults | eval ip_list="https://10.32.52.4,https://10.4.247.4,https://10.76.72.4,https://10.76.73.4,https://10.81.224.10,https://10.81.224.2,http://10.1.1.2,http://10.2.3.4,http://10.4.5.6" | makemv delim="," ip_list | mvexpand ip_list | rex field=ip_list "https?\:\/\/(?<ip>[\d\.]+)$" | lookup dnslookup clientip AS ip OUTPUT clienthost AS hostname | eval cmbd_name=coalesce(hostname,ip) | fields cmbd_name hostname ip ... View more