This provides a lot of info: index=_internal sourcetype=splunkd_access ( method=POST OR method=DELETE ) ( user!=splunk-system-user user!=- ) ( uri_path=/servicesNS/* uri_path!="*/user-prefs/*" uri_path!="/servicesNS/*/*/*/jobs/*/control" uri_path!=/servicesNS/*/mobile_access* ) | replace "*/ui/views*" with "*/ui_views*", "*/props*" with "**", "*/distributed/peers*" with "*/distributed_peers*", "*/server/serverclasses*" with "*/server_class*" in uri_path | where mvcount( split( uri_path , "/" ) ) > 6 | eval activity = case( method=="POST" AND like( uri_path , "%/acl" ) , "Permissions Update", method=="POST" AND NOT like( uri_path , "%/acl" ) , "Edited" , method="DELETE" , "Deleted" ) | rex field=uri_path "/servicesNS(/[^\/]+){3}/(?<object_type>[^\/]+)/(?<object_name>[^\/]+)" | eval object_name = urldecode( object_name ) | table _time, user, object_name, object_type, activity ... View more

Change the lookups (all 3 places) from global to App. ... View more

This would be on the server running the UF since it is doing the monitoring. ... View more

This is what I would do: On Cluster Master: splunk enable maintenance-mode Then follow the steps below on each indexer to delete data from an index... First come up with a search that finds only the data that you want removed. Run it to test that it works. Then make sure the admin user (or the user that that are using to do the delete) has can_delete role (or add the role). splunk edit user admin -role admin -role can_delete Run the search to delete the data (I want to delete data from the main index for the sourcetype syslog for a specific timeframe): splunk search 'index=main sourcetype=syslog | delete' -earliest_time '-26d@d' -latest_time '-3d@d' Then do the next indexer. Once you have run this on all indexers then from the Cluster Master: splunk rolling-restart cluster-peers splunk show cluster-status splunk disable maintenance-mode ... View more

You can try this if you are still having issues: Browse to /opt/splunk/var/lib/splunk/persistentstorage/fschangemanager_state/ and in that directory, there is also a file called "fschangemanager_state". Rename that file or move it to a temporary location and then restart Splunk. ... View more