This resolved my issue, thank you. Not sure what the root cause was, and have not seen this before. Steps to produce issue were: 1) yum update on RHEL 7 server 2) initiate reboot command (perhaps something here cause the issue? Splunkd did not completely stop?) 3) attempt to start splunk resulted in error: couldn't send sigterm to pid 332 operation not permitted 4) Resolution was to delete the /var/run/splunk/splunkd.pid as mentioned above, splunkd started on the next attempt ... View more

Thanks much! Going to try and use this to find hosts that are not reporting sysmon data. | tstats count WHERE index=* by host sourcetype | eval count=if(sourcetype="WinEventLog:Microsoft-Windows-Sysmon/Operational",1,0) | stats sum(count) as count by host #| search NOT (<exclude/filter out systems as needed here>, example: index=syslog) | where count = 0 | table host | rename host as "Hosts Not Sending Sysmon" ... View more

Yes, this was the issue I had as well running Splunk on 8.2.1 Splunk was unable to ingest syslog data while the application was running as the user Splunk. Running as root has resolved the issue. A more appropriate technical resolution would be identifying if we could set the user Splunk to be able to listen on ports lower than 1024 ... View more