All Topics

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Topics

I have a simple dashboard dropdown filter with two values - Yes, No (indicating whether a user is currently online or not). I have my searches working when a value is selected from the dropdown. Bu... See more...
I have a simple dashboard dropdown filter with two values - Yes, No (indicating whether a user is currently online or not). I have my searches working when a value is selected from the dropdown. But when nothing is selected the panel says "Search is waiting for input..." This seems like it would be a very common thing, but I can't find my answer anywhere in Splunk documentation or Splunk Answers.
I configured an alert when a VPN connection is established from an IP that is located abroad. Now I would like to test if the alert works as expected. What is the best way of doing this? Can I for ex... See more...
I configured an alert when a VPN connection is established from an IP that is located abroad. Now I would like to test if the alert works as expected. What is the best way of doing this? Can I for example copy a raw VPN login event, change the source IP, mark the event as alerttestevent and add it to Splunk to test the alert? Can this be automated somehow, i.e. when I adjust an alert I want to easily retest that everything still works as expected? I'm thinking about something like unit tests for Splunk alerts.
I am trying to understand how to set up Splunk for the first time. I have several Server VMs (exchange, DC, SCCM, Splunk) and about 70 workstations. I want to use Splunk to audit my workstations'... See more...
I am trying to understand how to set up Splunk for the first time. I have several Server VMs (exchange, DC, SCCM, Splunk) and about 70 workstations. I want to use Splunk to audit my workstations' event logs. This is a high level overview on how I understand that to occur: Install Splunk Enterprise on my Splunk Server (set up permissions as a domain user, etc) Configure Splunk Server as a Receiver Install a Universal Forwarder on every workstation and all of the other servers. Configure each Universal Forwarder- define inputs on the universal forwarder with configuration files Am I understanding this correctly?
Hello everyone, Do we have to create a eventgen.conf file for all apps. Or, does it automatically come under the relevant apps when the apps are installed? For example, i've installed SplunkAppFor... See more...
Hello everyone, Do we have to create a eventgen.conf file for all apps. Or, does it automatically come under the relevant apps when the apps are installed? For example, i've installed SplunkAppForFortinet with add-on and cisco_ios apps with add-on. There is a eventgen.conf file in the TA-cisco_ios/default but there is no eventgen.conf in the splunk_TA_fortinet_fortigate. Could you please help me to find what is the problem?
Hi, I have a requirement for forward 4 Windows Log EventCodes 4672, 4673, 4674 & 4624 to a destination from the HWFs, however exclude if these EventCode messages are for the "Logon Type=3" and forw... See more...
Hi, I have a requirement for forward 4 Windows Log EventCodes 4672, 4673, 4674 & 4624 to a destination from the HWFs, however exclude if these EventCode messages are for the "Logon Type=3" and forward rest everything. Expected result is something like: EventCode=4672 & LogonType= 3 : DISCARDED EventCode=4673 & LogonType= 3 : DISCARDED EventCode=4675 & LogonType= * : DISCARDED EventCode=4672 & LogonType= 2 : PROCESSED etc Does the below settings in the props and conf really help for this ? Thanks in advance. cat outputs.conf [syslog:syslog_destination] server=:514 type=udp disabled=false cat props.conf [WinEventLog:Security] TRANSFORMS-routing = routeDestination,excludeDestination cat transforms.conf [routeDestination] REGEX=EventCode=(4672|4673|4674|4624) DEST_KEY=_SYSLOG_ROUTING FORMAT=syslog_destination [excludeDestination] REGEX=Logon\sType:\t+3 DEST_KEY = queue FORMAT = nullQueue
Hi, since two days my splunk application is crashing 5 minutes after starting the application. In the Splunkd.log I get the following error description: 03-27-2020 13:00:16.418 +0100 ERROR S... See more...
Hi, since two days my splunk application is crashing 5 minutes after starting the application. In the Splunkd.log I get the following error description: 03-27-2020 13:00:16.418 +0100 ERROR StreamGroup - failed to drain remainder total_sz=59 bytes_freed=25106 avg_bytes_per_iv=425 sth=0x7fcb703feca0: [1585310416, /opt/splunk/var/lib/splunk/audit/db/hot_v1_27, 0x7fcb7570c650] reason=st_sync failed rc=-6 warm_rc=[-4,28] 03-27-2020 13:00:16.434 +0100 ERROR StreamGroup - failed to add corrupt marker to dir=/opt/splunk/var/lib/splunk/audit/db/hot_v1_27 errno=No space left on device 03-27-2020 13:00:16.454 +0100 ERROR BTreeCP - addUpdate: IOException caught: BTree::Exception: Record::writeLE failure in Node::_leafAddUpdate node offset: 24 order: 255 keys: 03-27-2020 13:00:16.466 +0100 ERROR BTreeCP - failed: failed to mkdir /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db/corrupt: No space left on device 03-27-2020 13:00:16.466 +0100 ERROR IndexWriter - failed to add corrupt marker to path='/opt/splunk/var/lib/splunk/audit/db/hot_v1_27' (No space left on device) I checked the space on the disk and the disk areas are only 28 % in use. So there should be enough space on the disk. Does anyone have an ideas what is the issue of this problem. Thank you
Hi all, I have a distributed multisite architecture, with a single Search Head, 2 indexers and, 2 Forwarders a Cluster Master, all in 7.3.3 I have to change some values in limits.conf to incr... See more...
Hi all, I have a distributed multisite architecture, with a single Search Head, 2 indexers and, 2 Forwarders a Cluster Master, all in 7.3.3 I have to change some values in limits.conf to increase the number of extracted fields. Where I have to change the value? In all machines? Only in the Cluster Master? Only in the Search Head? In the Search Head and in the indexers? I have read a lot of threads, and a lot of docs, but this is not explicitly documented, or i have not found it. Thanks in advance and sorry if this is a "noob" question, it's my first time
Hey, i have a Firewall Log and want to count the sending/receiving domains. My problem is that there is for one email three or more log entrys. Each message has a uniqe ID, which is availabl... See more...
Hey, i have a Firewall Log and want to count the sending/receiving domains. My problem is that there is for one email three or more log entrys. Each message has a uniqe ID, which is available in every associated log entry. Is it possible to merge these, so that i dont count an email twice or more. Thank you My current search looks like this: | rex field=_raw "to=<(?[\w\d\.\-]+\@(?[\d\w\.\-]+)\>)" | stats count(domain2) AS Anzahl by domain2 | rename domain2 AS "Domain Outgoing Anzahl" | sort - Anzahl | head 50
I am looking to upgrade Splunk Enterprise from 6.6.9 to 8.0.x. I understand this will take at least one intermediary step to Splunk 7.x. Splunk Enterprise Security 4.7.6 is also installed on the d... See more...
I am looking to upgrade Splunk Enterprise from 6.6.9 to 8.0.x. I understand this will take at least one intermediary step to Splunk 7.x. Splunk Enterprise Security 4.7.6 is also installed on the deployment and will require updating to remain compatible. The plan is to end up with SES 6.0.x. It seems that when upgrading SE the SES version should be compatible with the current version of SE and the version to upgrade to. https://docs.splunk.com/Documentation/Splunk/7.1.0/Installation/AboutupgradingREADTHISFIRST My problem then comes as I can't find a version of SES that is both compatible with 6.6.x/7.0 and 7.1+, according to the matrix on this page: https://docs.splunk.com/Documentation/VersionCompatibility/current/Matrix/CompatMatrix Am I interpreting the documentation correctly? And if so what possible workarounds could be used? Thanks
Hi and thank you in advance. I've simplified the problem for brevity sake. I'm trying to return multiple fields by way of using a subsearch. Looking for a recent match in index2 where there was a... See more...
Hi and thank you in advance. I've simplified the problem for brevity sake. I'm trying to return multiple fields by way of using a subsearch. Looking for a recent match in index2 where there was an older event occurring in index1. An example would be detecting an attack with previous reconnaissance. index=index1 earliest=-5h@h latest=-1h@h dst=* [search index=index2 earliest=-15m latest=now() dest=* | head 1 | eval index2_time=_time | return dst=dest ] This works, for finding a match. However, I want to pass up the _time of the more recent event in index2 (index2_time) and that doesn't appear to populate.
Hi, I have trouble to parse the timestamp of ESX-logs. The esx-syslog: Mar 18 21:15:02 hostname 2020-03-18T20:15:02.109Z hostname hostd-probe: info hostd-probe[FFA22350] and antoher log: ... See more...
Hi, I have trouble to parse the timestamp of ESX-logs. The esx-syslog: Mar 18 21:15:02 hostname 2020-03-18T20:15:02.109Z hostname hostd-probe: info hostd-probe[FFA22350] and antoher log: Mar 18 21:15:02 hostname 2020-03-18T20:15:02Z hostname hostd-probe: info hostd-probe[FFA22350] Because of some special multiline log I cut with SEDCMD the trailing splunk date "Mar 18 21:15:02". But this is done at the end of the parsing phase during indexing. So splunk tries first to read the date from the whole log. There are two formats, I like to have the date with milliseconds. Problems: the timezone is not recognized! I have an offset from 1 or 2 hours. And the milliseconds are not extracted. I tried: - TZ = UTC TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N -> does not work (maybe the T in the format string is not valid) - added some lines in datetime.xml and referenced it with DATETIME_CONFIG in props.conf (checked with btool ) <define name="_time_without_zone" extract="hour, minute, second, subsecond"> <text><![CDATA[(?<=T)]]></text> <use name="_hour"/> <text><![CDATA[:]]></text> <use name="_minute"/> <text><![CDATA[:]]></text> <use name="_second"/> <text><![CDATA[(?:(?: \d{4})?[:,\.](\d+))? {0,2}]]></text> </define> <timePatterns> <use name="_time_without_zone"/> <use name="_time"/> <use name="_time_without_subsec"/> <use name="_time_no_sub"/> <use name="_time_esxi_4x"/> <!-- Uncomment the below comments if ESX 4 exists in the environment <use name="_time_esx_4x"/> --> </timePatterns> SO how can I extract the correct date and timezone? Torsten
i have a column status, which will have a message saying "successful" or "unsuccessful", i want to color the cell green that has successful message else if unsuccessful color it red. please h... See more...
i have a column status, which will have a message saying "successful" or "unsuccessful", i want to color the cell green that has successful message else if unsuccessful color it red. please help me with the code
Hi, I am trying to understand how one can set multiple the passwords for individual proxy definitions in the website_monitoring app. Looking at https://lukemurphey.net/projects/splunk-website-mon... See more...
Hi, I am trying to understand how one can set multiple the passwords for individual proxy definitions in the website_monitoring app. Looking at https://lukemurphey.net/projects/splunk-website-monitoring/wiki/Using_multiple_proxies I only see multiple stanzas in website_monitoring.conf. In https://answers.splunk.com/answers/770852/website-monitoring-set-proxy-passwords-for-multipl.html I see references to the GUI for the passwords to get them encrypted. But the GUI only supports one default entry. So my question is, how to I set passwords for multiple proxy settings/users that are stored in encrypted form? thx afx
I cannot access my manage dashboard. In Splunk Enterprise 8.0.2.1 And Search Header Cluster. If you have seen 500 Internal Server Error like below, help me please. ==============error log====... See more...
I cannot access my manage dashboard. In Splunk Enterprise 8.0.2.1 And Search Header Cluster. If you have seen 500 Internal Server Error like below, help me please. ==============error log============= 2020-03-27 18:49:54,279 ERROR [5e7dcc420a7fb6f4323890] error:335 - Traceback (most recent call last): File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/cherrypy/_cprequest.py", line 628, in respond self._do_respond(path_info) File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/cherrypy/_cprequest.py", line 687, in _do_respond response.body = self.handler() File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/cherrypy/lib/encoding.py", line 219, in __call__ self.body = self.oldhandler(*args, **kwargs) File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/htmlinjectiontoolfactory.py", line 75, in wrapper resp = handler(*args, **kwargs) File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__ return self.callable(*self.args, **self.kwargs) File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/routes.py", line 383, in default return route.target(self, **kw) File "</SPLUNK/splunk_sh01/lib/python3.7/site-packages/decorator.py:decorator-gen-632>", line 2, in advancedsearch File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 40, in rundecs return fn(*a, **kw) File "</SPLUNK/splunk_sh01/lib/python3.7/site-packages/decorator.py:decorator-gen-630>", line 2, in advancedsearch File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 118, in check return fn(self, *a, **kw) File "</SPLUNK/splunk_sh01/lib/python3.7/site-packages/decorator.py:decorator-gen-629>", line 2, in advancedsearch File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 166, in validate_ip return fn(self, *a, **kw) File "</SPLUNK/splunk_sh01/lib/python3.7/site-packages/decorator.py:decorator-gen-628>", line 2, in advancedsearch File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 245, in preform_sso_check return fn(self, *a, **kw) File "</SPLUNK/splunk_sh01/lib/python3.7/site-packages/decorator.py:decorator-gen-627>", line 2, in advancedsearch File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 284, in check_login return fn(self, *a, **kw) File "</SPLUNK/splunk_sh01/lib/python3.7/site-packages/decorator.py:decorator-gen-626>", line 2, in advancedsearch File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/splunk/appserver/mrsparkle/lib/decorators.py", line 304, in handle_exceptions return fn(self, *a, **kw) File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/splunk/appserver/mrsparkle/controllers/admin.py", line 2995, in advancedsearch 'breadcrumbs': self.generateBreadcrumbs(namespace, 'advancedsearch'), File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/splunk/appserver/mrsparkle/controllers/admin.py", line 1381, in render_admin_template args = root.app.buildViewTemplate(namespace, ADMIN_VIEW_NAME, render_invisible=True, build_nav=False, include_app_css_assets=False) File "/SPLUNK/splunk_sh01/lib/python3.7/site-packages/splunk/appserver/mrsparkle/controllers/view.py", line 806, in buildViewTemplate viewList.sort(key=lambda x: x['label']) TypeError: '<' not supported between instances of 'NoneType' and 'str'
As the Splunkbase app "Cb Protection App for Splunk" is no longer supported and cannot be used, what is the best way to integrate Cb Protection into Splunk Enterprise? If I have to use CB Protection ... See more...
As the Splunkbase app "Cb Protection App for Splunk" is no longer supported and cannot be used, what is the best way to integrate Cb Protection into Splunk Enterprise? If I have to use CB Protection REST API, what should I do to integrate it?
I am searching for a list of regexes in a splunk alert like this: ... | regex "regex1|regex2|...regexn" Can I modify this query to get a table of the regexes found along with their count. Th... See more...
I am searching for a list of regexes in a splunk alert like this: ... | regex "regex1|regex2|...regexn" Can I modify this query to get a table of the regexes found along with their count. The table shouldn't show rows with 0 counts. regex2 17 regexn 3
Need to read from all files present in /temp/logs/ directory except one file abc.log Directory looks like xyz.log ab.txt ef.log abc.log inputs.conf [monitor:///temp/logs/] index = a... See more...
Need to read from all files present in /temp/logs/ directory except one file abc.log Directory looks like xyz.log ab.txt ef.log abc.log inputs.conf [monitor:///temp/logs/] index = abc_xyz sourcetype = server disabled = false
Hi, We use splunk-connect-for-kubernetes to send logs to splunk via HEC mechanism. Sending logs to splunk is fine, but searching is not. When we search for namespace=mynamespace "*Exception*" ... See more...
Hi, We use splunk-connect-for-kubernetes to send logs to splunk via HEC mechanism. Sending logs to splunk is fine, but searching is not. When we search for namespace=mynamespace "*Exception*" There is lots of missing logs, very few is returned. But, when I search like that: namespace=*mynamespace* "*Exception*" All is fine, all logs are returned Any suggestions? OUtput part of fluentd configuration: <match **> @type copy deep_copy true <store> @type splunk_hec protocol https hec_host "#{ENV['SPLUNK_HOST']}" hec_port "#{ENV['SPLUNK_PORT']}" hec_token "#{ENV['SPLUNK_TOKEN']}" host "#{ENV['NODE_NAME']}" source_key source sourcetype_key sourcetype <fields> pod namespace container_name container_id cluster_env cluster_name </fields> <buffer> @type memory chunk_limit_records 100000 chunk_limit_size 200m flush_interval 5s flush_thread_count 1 overflow_action block retry_max_times 3 total_limit_size 600m </buffer> <format> @type single_value message_key log add_newline false </format> </store> <store> @type prometheus <metric> (...) </metric> </store> </match>
Hello, Can anyone help me to find the issue and fix it? I need to grant permissions to use rest command to power role. I want to list users and roles assigned to them for monthly control purposes... See more...
Hello, Can anyone help me to find the issue and fix it? I need to grant permissions to use rest command to power role. I want to list users and roles assigned to them for monthly control purposes: | rest /services/authentication/users | fields title roles | rename title AS user | search roles IN (power admin ess_analyst) | stats values(roles) as roles by user The control performer is user who has a power role and when we are running the same query i am collecting much more entries than he is. What capability is missing? Power user capabilities: accelerate_search can_own_notable_events change_own_password dispatch_rest_to_indexers edit_analyticstories edit_glasstable edit_notable_events edit_search_schedule_window edit_sourcetypes edit_statsd_transforms edit_tcp edit_tcp_stream edit_timeline embed_report export_results_is_visible get_metadata get_typeahead input_file list_inputs list_metrics_catalog list_search_head_clustering output_file pattern_detect request_remote_tok rest_apps_management rest_apps_view rest_properties_get rest_properties_set rtsearch run_collect run_mcollect schedule_rtsearch schedule_search search search_process_config_refresh
Hi! Could you please help me with that special case of search? This is my data: User App 1. user1 appA 2. user1 appB 3. user2 appB 4. user1 appA If I would like to get the hits per user an... See more...
Hi! Could you please help me with that special case of search? This is my data: User App 1. user1 appA 2. user1 appB 3. user2 appB 4. user1 appA If I would like to get the hits per user and app by hour, i use the following | timechart span=1h count by app and now my question: I would like to have the events from the last 7 days and for each app i would like the max count(per Hour) for each day I have tried it with a second timechart after the first one and a span=1 but without success. Thank you for your help! Robert