Hey all,
I accidentally installed the latest Splunk add-on for Windows not realizing that it's not compatible with the Exchange app 3.4.1 (or 3.4.4). I reinstalled the add-on for windows v. 4.8.4 but now, if I do source="WinEventLog:Security"
in searches or reports, the searches work. But, if I do sourcetype="WinEventLog:Security"
it doesn't. The Exchange guided setup is looking for sourcetype="WinEventLog:Security"
and saying "sourcetype="WinEventLog:Security" | head 5"
did not return any events in the last 24 hours.
Any idea on where the source vs. sourcetype issue may reside? (Exchange app, add-on for windows, add-on for AD, add-on for DNS)?
My environment is a distributed search head/deployment server and an indexer and I think I have everything aligned but not sure...
Any guidance would be greatly appreciated!
Thanks!
This is probably because of Source and sourcetype changes for WinEventLog data in newer version which you installed accidentally.
Due to this change, events that have already been indexed (by latest version TA) will have WinEventLog
as sourcetype name. Add a wildcard to sourcetype in your reports and searches, so that splunk can search both version sourcetypes.
Hi pmac22,
Yes, this issue is because of sourcetype changes for WinEventLog data in latest Splunk add-on for Windows.
If you are getting data using Splunk_TA_windows latest version(v5.0.1) then query sourcetype="WinEventLog:Security" won't work.
Reason: In Splunk_TA_windows v5.0.x, Source types WinEventLog:* are merged and all the WinEventLog Data would be populated in a common "WinEventLog" Source type and would be differentiated by sources (which is unchanged).
That's why query (source="WinEventLog:Security) is working.
Solution: If you have reinstalled Splunk_TA_windows v4.8.4 then check if inputs are properly configured and props.conf and transforms.conf changes are reverted on forwarder as well as SH/IDX.
Note: The add-on Splunk_TA_windows v5.0.x is not compatible with Exchange app 3.4.x.
Thanks for the insights! I went to the eventtypes.conf file and found what you were alluding to. I feel like the splunk app for windows is just hosed with a combination of 4.8.4 and 5.0.1 references so I'm going to reinstall. Thanks again!
After doing more troubleshooting. I found out that because I renamed the Splunk_TA_windows directory on my indexer as a backup (Splunk_TA_windows_old) and never deleted it. It was totally causing confusion on the indexer.
Once I rm'd that old backup of the windows add-on, everything started working fine again. Just a pain that because of the incompatible windows add-on version, I have to be careful to remember to modify my queries to get historical data.
Thanks again for your assistance!!
This is probably because of Source and sourcetype changes for WinEventLog data in newer version which you installed accidentally.
Due to this change, events that have already been indexed (by latest version TA) will have WinEventLog
as sourcetype name. Add a wildcard to sourcetype in your reports and searches, so that splunk can search both version sourcetypes.
Ahhh, that makes sense. Is that a change in the ./local/props.conf file where I add the wildcard after the source (source*::) ?
Thanks again for the assistance!!
Splunk Windows Event Log
Apply the following properties to Splunk multi-line text files (.windows)
[source::....windows]
windows eventlog modular input sourceing
[source::WinEventLog://*]
TRANSFORMS-force_source_for_wineventlog_modular = force_source_for_wineventlog_modular,force_sourcetype_system_ias_f$
windows system sub-sourcetyping
[source::WinEventLog:System]
TRANSFORMS-force_sourcetype_system_ias_for_wineventlog = force_sourcetype_system_ias_for_wineventlog
Apply the following properties to all WinEventLog events
In addition to WinEventLog properties located in $SPLUNK_HOME/etc/system/default/props.conf
[source::(WMI:WinEventLog|WinEventLog)...]
Windows XML Event Log
[(?::){0}XmlWinEventLog:*]
All Windows Event Log
Apply the following properties to all Windows events
[source::(MonitorWare|NTSyslog|Snare|WinEventLog|WMI:WinEventLog)...]
Windows Security Event Log
[source::*:Security]
Windows System Event Log
All Windows System
[source::*:System]
I don't think changing props.conf will help.
If we think this through, you were using v4.8.4 for which sourcetype is WinEventLog://<application>|<System>|<Security>
then, the latest version for which all sources (app, sys, sec) are assigned one sourcetype WinEventLog
then rolled back to v4.8.4. So, you're having a mixture of old and new sourcetypes for the events.
My suggestion is continue searching on source="WinEventLog:Security"
in searches or reports or locate and modify savedseaches.conf (all your searches and reports and alerts are saved in this file) to include *
in sourcetype name inside searches.
Note: You might not find savedsearches.conf for some add-ons.
Thanks sudo!
Some of my reports/alerts don't reference source/sourcetype so i can't wildcard them but I took krakholiya's advice and took a peek in my eventtypes of the splunk app for windows and noticed that there are references to the updated wineventlog:security/application/system.
When modifying the searches based on the eventtype.conf info I was getting some of the searches to work but others not.
So I think my best option is to just reinstall the splunk app for exchange and all it's dependencies. it's a pain but at least I'll have more confidence in the setup afterwards. Now to decipher the exchange documentation black hole to ensure I put the right apps on the right host (search head, deployment server, index...)
Thanks again for your advice!