Hi All, It's already been determined that alarms/reports modifications are not being audited in _audit and _internal indexes. Reference to the links below.  @richgalloway can you recall the topic you gave me an answer to last week (second link below)?  https://ideas.splunk.com/ideas/E-I-49  https://community.splunk.com/t5/Alerting/Audit-changes-modifications-to-Alerts-Reports-using-REST-API/m-p/510793#M9352  Apart from keeping alerts/reports confs in Change Management System, the only option is (please correct me if i'm wrong) to use the bellow search that utilizes REST API and this is going to give us 1) time of modification 2) name of the alert/report in the form of uri and we can play further to extract this as a separate field and 3) who done it.    index="_internal" sourcetype="splunkd_ui_access" servicesNS file!="notify" method=POST   As you can see from above this would pick up changes made ONLY through the GUI but how about the CLI. If someone tampered with the savedsearches.conf file I'd like to audit those changes somewhere. Is there a straightforward way? I was thinking of file monitor of the file in Splunk and raise an alarm when something is changed but can't think of how I'll write the search query for the alarm. As am I novice to Splunk I read an article about diff command but found the documentation not clear and couldn't fully grasp how should I use it and if it's appropriate for my goal here? Also I read somewhere about putting the file in version control and monitor the changes but I hihgly doupt it that our customer would agree to that approach (don't ask). Any suggestions would be much appreciated. Thank you in advance.  ... View more

HI there, I am relatively new to Splunk but was given a task that I found very difficult. One of our customers is expecting an audit and has a variety of Reports and Alerts in one of their app. They would like to pop up an alert every time a someone modify any of the reports/alerts and the alert should give them what action has been done (e.g. modify, delete, add, remove) who done it, when and what has been changed (for instance if query's search is tampered with). I've tried everything that has been posted here and it's always close but no cigar. It seems that the internal logs of _audit and _internal indexes do not log these changes. In the end I came up to REST API. What I did is as follows: 1)  I typed the search below to get the alerts/reports         rest splunk_server=local /servicesNS/-/{app_name}/saved/searches | fields title search eai:acl:owner eai:acl:app alert_type updated cron_schedule auto_summarize.suspend_period dispatch.earliest_time dispatch.latest_time id          2) I exported the results as a csv file and renamed the search column to oldSearch 3) I imported back the csv (compareSearches) as lookup and used the following query         | rest splunk_server=local /servicesNS/-/{app_name}/saved/searches | fields title search eai:acl:owner eai:acl:app alert_type updated cron_schedule auto_summarize.suspend_period dispatch.earliest_time dispatch.latest_time id | join [| inputlookup compareSearches.csv | table title oldSearch] | where search!=oldSearch         4) That almost gave me what I wanted but not exactly.  This search only catches changes in the reports/alerts queries. But if report/alert gets deleted or the schedule time is changed. 5) The below line would create an additional fields with info regarding what type of change occured.         | eval changeType = if(search!=oldSearch, "Query changed", "Other change occured")         6) So at the end I still don't know who did the action and apart from changing the query I stuck on the rest of the changes and how can I display them.  I read all post regarding similar cases but nothing worked for me. Any help would be much appreciated. Thank you.   p.s. At first I used one account to create a report and to modify it and the ID field gave me something like https://127.0.0.1:8089/servicesNS/{mu_name}/a1siem/saved/searches/{my_report} However when using another account it gives back  https://127.0.0.1:8089/servicesNS/nobody/a1siem/saved/searches/{my_report} ... View more