I want to search Splunk logs in order to see changes to Splunk Objects by user. An example would be to see an event which reads something like the following:
date=1/1/2000, time=08:00:00.000, object=app, object_name=app1, file_name="local.meta" action=permissions_change, value_new="export=system", user_splunk=user1 date=1/1/2000, time=08:01:00.000, object=fields, object_name=sourcetype1, file_name=props.conf, action=line_added, value_new="TRANSFORMS-nullqueue_pound = nullqueue_pound", user_splunk=user1
.. or logs that reveal similar information along that effect.
What search reveals the changes in Splunk objects by the user that made each change?
Here is what I have for the part on user changes to Splunk knowledge objects:
index=_internal sourcetype=splunkd_ui_access source="/opt/splunk/var/log/splunk/web_access.log" OR source="/opt/splunk/var/log/splunk/splunkd_ui_access.log" host="host_splunk_server" NOT (sid OR GET OR 127.0.0.1) | search uri_path!="/en-US/debug/refresh" uri_path!="/en-US/account/login" | table _time user clientip method uri_path web_language web_menu web_page web_app object_type web_dir object_name | rename clientip AS src
The internal logs do not contain the exact change made in many cases. The following elements are revealed together in this search: app installs, app and saved search (among many others KO) permissions changes, and logging into and out of the splunk web client. I have an extraction that extracts the uri_path and a transforms that splits the uri_path into the fields listed in the search. From there, btools can pull the appropriate conf file, but an exact connection to the exact change with the conf file would not be possible with the available information unless a change was made in the conf file to exactly one KO since the last time btools was used to record that conf file for a diff comparison to be possible.
Here are the transforms that I am using:
source key: uri_path regex: \/(?<splunk_language>[\w\-]*)\/?(?<splunk_menu>\w*)\/?_*(?<splunk_page>\w*)(\/servicesNS\/\w*)?\/?(?<splunk_app>\w*)\/?(?<ko_type>\w*)\/?(?<splunk_dir>\w*)\/?(?<ko_name>[\w\.]*) source key: other regex: - (\w+) (\w+) format: session_id::$1 duration::$2 auto clean names
Oh that would be so very nice, but as of 6.2 at least, I don't know of anything that tracks such out of the box... you may be able to look at splunkd_access logs, but that's not going to tell you how things have changed. There are a number of folks that use various source control and configuration management mechanisms instead, but that's a more rigid system.
I have a way to track changes to config files. I have a separate way to track when users make changes. But correlating the two would require very careful integration of the two searches in a very smart way. Still working on it.