How do I successfully change the Splunk instance (all in one indexer, search head, webUI etc) VM's IP address? Since Splunk 7 - 8, the Splunk install automatically uses the VMWare admin interface making it so I can only log into Splunk via the VM. I follow the Bind IP instructions here : https://docs.splunk.com/Documentation/Splunk/6.1.3/Admin/BindSplunktoanIP to get it to where I can now log into the webUI via the IP I specify, but this breaks Splunk. Upon reboot I get a 500 service not found. IP and ports are as specified when I reboot Splunk, but the webUI will no longer work and I have to restore from a snapshot. ... View more

Updated Splunk 6.5.x to 7.3.0 and now one of my main dashboards has, "Error parsing dashboard XML: malformed URI sequence. Go to "Edit Source" to fix." Going into "Edit source", it states "Error on line -1: malformed URI sequence" First few lines of XML: `` > `<form> <label>Overview - > v2.4</label> <fieldset > submitButton="false" > autoRun="true"> > <input type="time" > searchWhenChanged="true"> > ... View more

I am noticing that Splunk ingestion is spotty. For example, out of a hundred machines that have pluginID 38153 results a few days ago (verified in the SecCenter GUI), only three of these machines/results are found in Splunk. Are there a limits.conf or another setting that needs to be changed to allow full ingest? ... View more

Trying to figure out a string to find open windows locked-screen sessions Monitored all security events when doing a log on, full log-off and locked screen 4624 logon (type7 = logon from a locked screen) 4624 logon (type 2 = full logon when no active session running) 4634 = locked screen 4647 = full log off 4673 = privileged service called – this one is interesting, there is a 4673 heart beat on the machine that has a locked screen user session. Doing the below Index=winevents EventCode=4624 OR EventCode=4634 | transaction host startswith=EventCode=4624 endswith=EventCode=4634 | table what you want Problem with the above — it finds the log-on and then locked screen events, but it isn't catching after that event. Then user logged on again, then fully logged out — so you see, every time someone did a log on and locked screen — even if they later logged in yet again and the legit logged off cleaning the situation. This is where maybe that 4673 event ID is of value? ... View more

I've seen searches using _internal to identify OS, but is there a way to identify what clients are physical and which are VMs? ... View more

I have a different (and mislabeled) post about this that is labeled answered, but the issue is not. I don't know if I found a bug- or more likely I'm really bad at hunting down a simple issue, but I found if my router/switch goes down (APC took a dump) for a few hours, upon restoring the network the A3Sec app (for pfsense logs) will start to get the following type of errors in the splunkd.log: 06-27-2018 19:23:58.543 -0700 WARN DateParserVerbose - A possible timestamp match (Sat Setp 8 18:46:43 2001) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Contex: source=udp:514 | host xxx.xxx.x.x | pfsense_syslog | 06-26-2018 20:51:26.834 -0700 WARN DateParserVerbose - Failed to parse timestamp in the first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to time stamp of previous event (Tue June 26 08:27:00 2018). Context: source=udp:514 | host =xxx.xx.x.x | pfsense_syslog This is on a CentOS7 VM, ESXi 6.4. The CentOS7 box gave itself an IPv6 address when the network went down, had to get it back to it's IPv4 when the network got back up. I reviewed the app's props.conf, transforms, multiple reboots, deleted the gw_pfsense index, made it again, clean indexes (I guess this is clearing the fishbucket on 7.x?), turn pfsense syslog output on and off, reboots etc to no avail. Ultimately I went back to an earlier snapshot and now gw_pfsense is indexing firewall events again. ... View more

How do you go about ensuring splunk forwarders forward all data from a gold image created VM that then gets blown away when a user logs off? How are people managing VM creation/destruction and in relation to the monitoring console "missing forwarder" report growing as VMs come and then go? ... View more

I have a CentOS7 Splunk 7.x build using the A3sec pfSense app, snort for splunk app and missile app. the A3sec pfSense app does not resume ingesting logs if the VM has been down/restarted. I ensured: - not firewalld - tcpdump shows the syslog is flowing in on UDP 514 - Rebuilt the 514 data input - Restarted the VM and splunk service (this will cause just a brief grab of a few logs as they flowed in) - Restarted the pfSense router (this fixed that the snort logs on UDP 1514 were having the same issue) I even disabled firewalld for trouble shooting sake. What other steps should I take? ... View more

Hello, I'm running the A3Sec pfSense app (great setup and I threw the missile map in, outstanding stuff) and since going to Splunk 7.x.x the panels' searches seem to time out around 5 seconds, so I am not getting the full range of results. I must not be using the correct terms because I could not find any threads in relation to changing this setting (how long panels are given to run their search until it times out). ... View more