Hi,   I am hoping to get some help in creating a search, which will be turned into an alert - I am working with system logs from a monitoring device, where a log is submitted when any one of ~600 servers go down and while the server stays down a new log is dropped every ~10 mins, then if the server comes back up a "Reconnect" log is submitted. I am wanting to get the search to return me the name of a server/agent that has had at least 1 "disconnect" but no "reconnect" entry within a time period and then once a reconnect is received - the server is no longer listed. I am not very experienced with Splunk and thus far only have a search that is returning me counts of both types of events (connect/disconnect): index="XXXlogs" sourcetype="systemlog" eventid="*connectserver" devicename="device1" logdescription="Agent*" |  stats count by win_server, event_id Any help is appreciated.

Hi, I just installed the Configuration explorer in order to edit my transforms.conf. First I edited the settings file to set write-access = true . I then restarted splunk. When I now try to edit the transforms.conf I cant save my changes. An error message appers saying: This file cannot be saved. Is there another way to edit the file or how can I enable writing via the conf explorer?

Hello together, we moved our data to a new index cluster and since then we are unable to delete events with the "| delete" query. We have an test system, which is a single server instance that will execute the same query. Datasets are identical on both systems. Heres a sample command we are trying to run on our clustered server: index=name1 sourcetype=type1 earliest_time=-3d | delete Since the documentation also noted that sometimes you should eval the indexname to delete events, we also did that index=name1 sourcetype=type1 earliest_time=-3d | eval index=name1 | delete   Both queries without the delete command only return a small set of 8 events. If we pipe the result to "delete", then there's no error message or warning. However the returned result table shows that zero files have been deleted. Currently we do have a new search cluster and also our old single search head connected to this index cluster. The old single searchhead was previously also the single instance where we migrated our data from to the new index cluster. Despite that migration nothing has been changed on that servers user/role configuration. Still delete is not working anymore on that search head too.   We did follow all instructions on the splunk documentation to ensure that it is not a configuration problem https://docs.splunk.com/Documentation/SplunkCloud/latest/SearchReference/Delete   Additionally we did the following to troubleshoot the delete process: We tried other datasets/indexes on our cluster server -> same result (working on test server) We checked that our user has “can_delete” roles + created new local users with “can_delete” role Both without success. We also noticed that if the user has no “can_delete” role assigned, the query result will also notify that permissions are missing Since we don’t get that message, we believe that the role is set correctly We compared the authorize.conf from our test and cluster system and didn’t see any differences for those roles We checked all servers splunkd logs after sending the delete command and no information/errors are available We checked that on the file system the bucket folders/files have the correct access permissions (rwx) for “splunk” user - We restarted the index cluster We tried the search query directly on the cluster master, on each search head cluster member and on the old single search head of our clustered system We ran splunk healthcheck with no issues We checked bucket status for the index cluster We checked monitoring console for indexers with no issues We ran | dbinspect for the index and checked if the listed filesystem paths are accessible by the splunk user We ran the search queries in the terminal via “splunk cli”, with no errors or additional messages being shown Both test and cluster servers are running on the same version (8.1.6) The data from the query was also indexed far after the migration

Trying to implement an alert on detecting spikes in logged events in our Splunk deployment and not sure how to go about it... For example: Have 15 hosts with varying levels of sources within each... one of my sources in a host averages about 5-6k events per day over the past 30 days; then out of the blue, we're hit with 1.3 million events on one of the days. Assuming the alert would need to be tailored to each host (or source, not sure) and would need an average number of events over a "normal" week to compare to when there's a spike? Any help would be greatly appreciated.  

Hello Splunkers!!   When i am upgrading my web HF from 8.0.0 to 8.1.2 then i am getting below error. Please let me what is available workaround for the below issue.   The TCP output processor has paused the data flow. Forwarding to host_dest=<indexer_name> inside output group splunkcloud from host_src=<HF_NAME> has been blocked for blocked_seconds=1970. This can stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data.

Hello, My team is interested in using Grand Central but we'd like to compare the templates generated by Grand Central to Trumpet to see if there are any significant differences before making the switch. We haven't been able to get the app to work when importing it to our search head, so I'm hoping you can provide something generic. We are trying to capture the following sources if we use Grand Central: CloudTrail Config Notifications GuardDuty VPC Flow Logs If you need anymore information, please let me know. Best, Faleon

I use Ansible to install and configure Splunk Universal Forwarder on multiple servers.  However, it's difficult to maintain a link to the product, the links are not consistent and contain unpredictable data, such as the git commit for the version.  Is there a link that would allow me to generate a link dynamically only knowing non-changing properties of the download or predictable values, such as version, platform, package type, etc.  For example: "https://download.splunk.com/products/universalforwarder/releases/{{ splunk_version }}/linux/splunkforwarder-{{ splunk_version }}-{{ splunk_os }}-{{ splunk_arch }}.{{ splunk_pkg }}" or to get the latest version something simple like: https://download.splunk.com/products/universalforwarder/releases/latest  

I had encoutered an interesting question from my client/security SME 1. Which one is better. To have Splunk Security Essentials or to retain Enterprise Security + Content updates? 2. Where are the detection rules kept in Splunk Security Essentials kept?   As far as I understand the Splunk ES content update is quite easy to understand and we can customise the savedsearches.conf (rules) to fit our environment. On other hand, Splunk security Essentials, we couldn't figure out where the rules exist and modify them. Any ideas how to get the detection rules of Splunk Essentials? Also what would be the future direction of these developments? wanted to stick to one of them if possible

The following do not give the IP for the Splunk Enterprise Security (ES). Is there a better SPL to provide the list of all Splunk instances names, IPs. Specially the ES? Thanks a million in advance.   | rest /services/server/sysinfo splunk_server=local | table splunk_server | rest /services/server/sysinfo splunk_server=local | table splunk_server | lookup dnslookup clienthost as splunk_server OUTPUT clienthost as ipAddress  

Hello all  I'm using Splunk Cloud Platform and i want to know how to acces to this URL. URL : /splunkd/__raw/services/data/lookup_edit/lookup_contents I saw it there https://lukemurphey.net/projects/splunk-lookup-editor/wiki/REST_endpoints but i don't really understand how it works.  If someone can help me ! @LukeMurphey @Anonymous  Thank you all

Hello,   We received data from Alicloud and found there are alot of duplicate fields populate in Interesting fields like source , source_ . Is there any query i can use to check how many fields and events are duplicate?