_raw data [fw4_deny] [ip-address] start_time="1998-07-07 11:21:09" end_time="1998-07-07 11:21:09" machine_name=test_chall_1 fw_rule_id=11290 fw_rule_name=auto_ruleId_1290 nat_rule_id=0 nat_rule_name= src_ip=1xx.1xx.0.x user_id=- src_port=63185 dst_ip=192.168.0.2 dst_port=16992 protocol=6 app_name=- app_protocol=- app_category=- app_saas=no input_interface=eth212 bytes_forward=70 bytes_backward=0 packets_total=1 bytes_total=70 flag_record=S terminate_reason=Denied by Deny Rule is_ssl=no is_sslvpn=no host=- src_country=X2 dst_country=X2 [resource_cnt] [10.10.10.10] time="1998-07-07 11:24:50" machine_name=test_boby_1 cpu_usage=7.0 mem_usage=19.8 disk_usage=5.6 cpu_count=32, cpu_per_usage=3.0-2.9-2.0-2.0-2.0-2.0-0.0-0.0-23.0-7.9-7.0-6.9-19.4-19.0-8.0-7.0-1.0-1.0-16.0-1.0-2.0-2.0-1.0-2.0-24.8-9.0-16.2-8.0-9.0-9.9-5.0-8.1 my props.conf [secui:fw] DATETIME_CONFIG = LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true SEDCMD-duration = s/duration=\d+\s// SEDCMD-fragment_info = s/fragment_info=\S*\s// SEDCMD-ingres_if = s/ingres_if=\S*\s// SEDCMD-input = s/input\sinterface/interface/ SEDCMD-packets_backward = s/packets_backward=\S*\s// SEDCMD-packets_forward = s/packets_forward=\S*\s// SEDCMD-pre = s/^[^\[]+// SEDCMD-terminate_reason = s/\sterminate_reason=-// SEDCMD-user_auth = s/user_auth=\S*\s// SEDCMD-userid = s/user_id=\S*\s// TRANSFORMS-secui_nullq = secui_nullq TRANSFORMS-stchg7 = secui_resource TRANSFORMS-stchg8 = secui_session category = Custom description = test disabled = false pulldown_type = true <Fields you want to exclude> fw_rule_name, app_saas nat_rule_name, is_ssl user_id, is_sslvpn app_name, host app_protocol, src_country app_category, dst_country I want to exclude fields that I want to exclude from being extracted at index time. Currently, fields that I want to exclude are automatically extracted when searching for fields of interest. Is there a way to do this?   ... View more

I want to use the splunk app for lookup file editing to export a csv lookup file and automatically apply utf8-bom encoding when opening the downloaded file on my local PC. How can I do this? I went into that path and modified lookup_editor_rest_handler.py and added a utf8-BOM stream to csv_data, but that setting is not applied. When the user opens the exported Excel file on a local PC, a problem occurs where non-English characters are encoded incorrectly. $SPLUNK_HOME/etc/apps/lookup_editor/bin/lookup_editor_rest_handler.py     import codecs def post_lookup_as_file(self, request_info, lookup_file=None, namespace="lookup_editor", owner=None, lookup_type='csv', **kwargs): self.logger.info("Exporting lookup, namespace=%s, lookup=%s, type=%s, owner=%s", namespace, lookup_file, lookup_type, owner) try: # If we are getting the CSV, then just pipe the file to the user if lookup_type == "csv": with self.lookup_editor.get_lookup(request_info.session_key, lookup_file, namespace, owner) as csv_file_handle: csv_data = csv_file_handle.read() csv_data = codecs.BOM_UTF8.decode('utf-8')+csv_data # If we are getting a KV store lookup, then convert it to a CSV file else: rows = self.lookup_editor.get_kv_lookup(request_info.session_key, lookup_file, namespace, owner) csv_data = shortcuts.convert_array_to_csv(rows) return { 'payload': csv_data, # Payload of the request. 'status': 200, # HTTP status code 'headers': { 'Content-Type': 'text/csv; charset=UTF-8', 'Content-Disposition': f'attachment; filename*=UTF-8\'\'{lookup_file}' }, } except (IOError, ResourceNotFound): return self.render_error_json("Unable to find the lookup", 404) except (AuthorizationFailed, PermissionDeniedException): return self.render_error_json("You do not have permission to perform this operation", 403) except Exception as e: self.logger.exception("Export lookup: details=%s", e) return self.render_error_json("Something went wrong!")       ... View more

indexes.conf [volume:hot] path=/mnt/splunk/hot maxVolumeDataSizeMB = 40 [volume:cold] path = /mnt/splunk/cold maxVolumeDataSizeMB = 40 [A] homePath = volume:hot/A/db coldPath = volume:cold/A/colddb maxDataSize = 1 maxTotalDataSizeMB = 90 thawedPath = $SPLUNK_DB/A/thaweddb [_internal] homePath = volume:cold/_internaldb/db coldPath = volume:cold/_internaldb/colddb thawedPath = $SPLUNK_DB/_internaldb/thaweddb maxDataSize = 1 maxTotalDataSizeMB = 90 I collected data from each index, and the percentage stored in cold volume was A=30MB, _internaldb/db=10MB. This was understood to account for a larger percentage because the data volume and collection speed of the A index was larger and faster than that of _internal collection. If you stop collecting data from the A index and maintain data collection only for the _internal index, the old buckets in _internaldb/db will be moved to _internaldb/colddb in the order they were loaded in _internaldb/db, and will not be maintained in colddb in the order in which they were loaded in _internaldb/db, but will be immediately deleted. Additionally, data that existed in A/colddb is deleted in oldest order. I understood that the cold volume is limited to 40 and the cold volume is already full, so it will not be maintained in _internaldb/colddb and will be immediately deleted. However, why is the data in A/colddb deleted? Afterwards, when the A/colddb capacity reaches 20, A/colddb is not deleted. The behavior I expected was that if A/colddb capacity is deleted until it becomes 0, the old buckets in _internaldb/db would be moved to _internaldb/colddb and then maintained. I'm curious why the results are different from what I expected, and if maxTotalDataSizeMB is the same, the Volume maintains the same ratio. ... View more

I want to create a custom role to manage splunk risky commands. I looked for configuration files related to risky commands and found that it is related to web.conf command.conf file and I found that you can disable risky commands by setting [command] is_risky=false in command.conf file. What I want is to give a role to manage risky_command so that I can get different search results than other users. I wonder if it is possible to create such a role. ... View more

My friend and I have the same indexes.conf, but why are the bucket sizes being created different? Mine is around 1MB, but my friend's are created in 5.x MB units.. indexes.conf [volume:hot] path = /data/HOT maxVolumeDataSizeMB = 100 [volume:cold] path = /data/COLD maxVolumeDataSizeMB = 100 [lotte] homePath = volume:hot/lotte/db coldPath = volume:cold/lotte/colddb maxDataSize = 1 maxTotalDataSizeMB = 200 thawedPath = $SPLUNK_DB/lotte/thaweddb     ... View more