Hello, With these sorts of issues it's best to work your way down to eliminate the possible causes. Take an exemplar broken search from the dashboard and try to run it manually: eventtype=msad-successful-user-logons   If that doesn't work try to run the definition manually: eventtype=wineventlog_index_windows eventtype=wineventlog_security EventCode=4624 user!="*$"   If that works, make sure the msad-successful-user-logons definition is correct and shared properly. If not, try expanding your index eventtype: (index=msad OR index=main) eventtype=wineventlog_security EventCode=4624 user!="*$"   If that works, make sure your definition is correct and shared properly. If not, try expanding the wineventlog_security eventtype: (index=msad OR index=main) (search = source=WinEventLog:Security OR source=WMI:WinEventLog:Security OR source=XmlWinEventLog:Security) EventCode=4624 user!="*$"   If that works, make sure Splunk_TA_windows is installed the wineventlog_security eventtype is working. If that doesn't work then your problem is not with the eventtype definitions, but rather with the data itself. Things to try:   Do you have Splunk_TA_windows installed on your indexers/search heads? Are the source's renamed correctly as per TA_Windows ta-windows-fix-xml-source definition and the requirements of the wineventlog_security eventtype? Are your indexes correct and populated within the search timeframe? Finally, if you still can't get results, try stripping of key values from the search to check if the search is working: (index=msad OR index=main) (search = source=WinEventLog:Security OR source=WMI:WinEventLog:Security OR source=XmlWinEventLog:Security) If you get results, the problem is with the field extractions: EventCode=4624 user!="*$" check that Splunk_TA_windows is working as expected, check your inputs, props and transforms are all aligned. Good luck! ... View more

Hi, The wording is quite tricky but I will do my best to explain: 1) The maximum number of concurrent historical scheduled searches on this cluster has been reached: Platform-level. Essentially,  the Splunk platform has reached its concurrent search limit as defined by the concurrency settings in limits.conf. For example, if the limit is five, you might have six different searches all scheduled at once, triggering this error. 2) The maximum number of concurrent running jobs for this historical scheduled search on this cluster has been reached This is defined on a per-search level under max_concurrent within savedsearches.conf (advanced search settings). It is one by default. This means that one particular search has been scheduled whilst previous instances of that same search is still running. So for example, you schedule a particular search to run every two minutes but it takes six minutes to run. Splunk starts it, and two minutes later, goes to schedule it for a second time but its still running, throwing the error. As a tip, 1) is typically caused by incorrectly scheduled searches all scheduled for the same time. Whereas 2) is caused by a particular search being scheduled too frequently or running for longer than expected, causing it to overlap with itself. ... View more

Hi, It sounds like you've made great progress, nice one. There are multiple designs and opinions out there regarding getting syslog into Splunk. It's up to you to decide what's best. To get you started there are tools such as Splunk Connect For Syslog which provides an "all in one" feel, you can also use a syslog service such as rsyslog or syslog-ng to listen for your logs and cache them to disk and then forward them via a monitor stanza in inputs.conf. However, if you want Splunk to listen directly, here is an example inputs.conf that you can tweak for your deployment:   [udp://10514] disabled = false connection_host = ip sourcetype = <<firewall_product>> index = main   For sourcetype, look on Splunkbase for your firewall vendor to check if there is an appropriate TA that you can use for field extractions. For example palo-alto firewall would be pan_log.  For index, pick an appropriate index to suit your needs. Finally, inputs.conf can either be deployed within an app (recommended) or directly under /opt/splunk/etc/system/local/ Also, make sure that 10514 is permitted on the local firewall. ... View more

Hi, You can do that with an eval command. | eval firstSeenTS = strptime(firstSeen, "%b %d, %Y %H:%M:%S %Z"), lastSeenTS = strptime(lastSeen, "%b %d, %Y %H:%M:%S %Z"), firstLastDiff = (lastSeenTS - firstSeenTS)/86400, firstNowDiff = (now() - firstSeenTS)/86400 If you want to round your days down to whole numbers you can use floor() ... View more

The issue was actually the "case_sensitive_match" transforms.conf entry. The case_sensitive_match entry requires the kvstore contents to be lower case. If you don't drop the case on your kvstore contents then the | lookup command will stop matching, even if the case matches! This is one for my notebook... ... View more