Thanks, I think https://docs.splunk.com/Documentation/Splunk/9.2.1/InheritedDeployment/Ports is the one included? ... View more

Hey @naisanza,  I installed the splunk on the Ubuntu (WSL) and encountered the same issue and i have tried the option you have provided and it worked.. Thank you very much.  ... View more

Do not modify default folder instead use local 🙂 ... View more

If done correctly, multi-line events will be indexed as a single event and duplication is an unlikely problem. ... View more

Thank you! My forwarders should already be pointed at a hostname and not an IP. I think what I'll do is be slightly messy and once the old Indexer is gone I'll recycle its A record and turn it into a CNAME that I'll point at the new server. I can also re-IP the new server for good measure. Thanks too for updating the documentation, that will be most helpful. ... View more

You have vestiges of a previous install where you previously created user = admin . Try the password changeme . If that doesn't work, the easiest way to fix it is to uninstall and then reinstall. Upon reinstall, you will be prompted to create a new admin user. Cake. ... View more

Hi, I confirmed with an engineer that this option will work in the way it's been depicted here. ... View more

I think this is an issue with that server. The forwarder service is not appearing in the list of services yet the process for the service is running. I will focus on resolving that issue first. Thanks for your help. ... View more

This message was displayed after attempting to launch Splunk Enterprise 6.5 via a Remote Desktop Connection from one VM Server to another. (I do not have a screenshot available). The issue was resolved by setting the screen resolution to one of the pre-set values, in this case, 1152x864. ... View more

Hmmm. I am surprised to hear about a deployment where that file doesn't exist. Is this merely a scenario of $SPLUNK_HOME not already defined? Did you change that to where Splunk is installed (like /opt/splunk/ )? ... View more

I currently run 14 indexers over ~250 spinning disks with raidz2. I've been running splunk over zfs for over 3 years. No problems, in fact I initially ran indexers with several different backends and zfs was the most stable. My main indexer is a 1U supermicro box with a cost of ~5k. Box has 12 spinning media drives (raidz2), used Hitachi Ultrastar drives. 2 SSD's for bootup disks. Then install a PCI NVMe drive for your cache drive. (2-4 or so gigabytes per second speed). I'd recommend altering the default settings of zfs so it'll be more aggressive with using the NVMe drive. Get at least 12 cores in order to remove the annoyance about Enterprise Security complaining <12 cores. ... View more

Well yea.., I know that NOOooOOowww... 😉 Seriously, thanks again for pointing me in the right direction. ~Ed ... View more

on the forwarder ... View more

Technically it's not a known issue, but a reversion to a previous behavior that was not caught in the documentation. It has been added now. Again, sincere apologies for the headache that this caused. ... View more

I was experiencing this problem as well and it ended up being a permissions issue on the mongo folder itself. after reading through these posts as well as the referenced posts I did the following: 1) stopped splunk process 2) right clicked on mongo folder located in C:\Program Files\Splunk\var\lib\splunk\kvstore 3) selected security and altered the security permissions for local admin as well system, before exiting I selected the replace child object permissions entries with inheritable permissions entries from this object. 4) restarted splunk and my errors went away. these steps cleared the error message Key value store must be enabled. Please enable it" and it enabled me to perform a KVStore migrate which i was not able to do before fixing this error. (KV Store initialization has failed.) ... View more

They are inbound, meaning that the Splunk process listens for connections from other hosts on these ports. ... View more

I ran this query to try to figure this out too. earliest=-7d eventtype=* | stats count by eventtype | Table eventtype, count eventtype count external-referer 188526 external-referer perfmon perfmon-index perfmon_logicaldisk perfmon_windows visitor-type-referred windows_performance 0 external-referer perfmon perfmon-index perfmon_windows visitor-type-referred windows_performance 0 external-referer perfmon-index visitor-type-referred wmi_windows 0 external-referer summary-internet-mail visitor-type-referred 1653 external-referer summary-mailbox-size summary-user-population visitor-type-referred 2408 external-referer summary-user-mail visitor-type-referred 3176 perfmon 177706 perfmon-index 187500 perfmon_logicaldisk 7777 perfmon_windows 177706 visitor-type-referred 188526 windows_performance 177706 windows_updatelog 1026 windows_updatelog_status 16 windowsupdatelog_windows 1026 wmi_windows 13756 I set my index to MSAD when I set up forwarders on all my DCs (see Settings--> Data Inputs --> Remote Eventlog Collections) But there do not appear to be any eventtypes = msad-successful-user-logons or msad-dc-health as many splunk AD infrastructure documents suggest. There must be some setting that maps these events ID to "eventtypes" . I think there is a -nt6- now in between msad-account-lockout 8 msad-account-unlock 2 msad-ad-access 56 msad-admin-audit 18 msad-computer-changes 7 msad-disabled-logons 787 msad-dns-events 12 msad-failed-computer-logons 79 msad-failed-user-logons 800 msad-group-changes 1 msad-groupmembership-changes 1 msad-nt5-successful-user-logons 210 msad-nt6-account-lockout 8 msad-nt6-account-unlock 2 msad-nt6-ad-access 56 msad-nt6-computer-changes 7 msad-nt6-computer-changes 7 msad-nt6-disabled-logons 787 msad-nt6-failed-computer-logons 79 msad-nt6-failed-user-logons 800 msad-nt6-group-changes 1 msad-nt6-groupmembership-changes 1 msad-nt6-password-changes 4 msad-nt6-successful-computer-logons 166356 msad-nt6-successful-user-logons 60594 msad-nt6-user-changes 8 msad-password-changes 4 msad-successful-computer-logons 166356 msad-successful-user-logons 60804 msad-user-changes 8 ... View more

I had the some issue with: CentOS release 6.2 (Final) Unix addon version 5.2.4 After some long hours I was able to successfully forward data by changing /opt/splunkforwarder/etc/apps/Splunk_TA_nix/bin/vmstat.sh Removed the last lines: $CMD | tee $TEE_DEST | $AWK "$HEADERIZE $MASSAGE $FILL_BLANKS $PRINTF" header="$HEADER" echo "Cmd = [$CMD]; | $AWK '$HEADERIZE $MASSAGE $FILL_BLANKS $PRINTF' header=\"$HEADER\"" >> $TEE_DEST And replaced them by: RESULT=$($CMD | tee $TEE_DEST | $AWK "$HEADERIZE $MASSAGE $FILL_BLANKS $PRINTF") echo $RESULT | awk -F'pgPageOut_PS ' '{printf "%s\n%s", $1, $2}' And it worked! Note: At the moment from my investigation it seems there is some kind of timeout on script stanzas that we are not able to control in the configuration file of the universal forward. ... View more

I downvoted this post because doesn't answer the question. should be move to comments ... View more

The Splunk Add-ons for Microsoft Exchange (TA_Exchange_* - included in the Splunk App for Microsoft Exchange) collect data from the Exchange servers. This data must be covered by the Splunk App for Microsoft Exchange license. It is separate from any other data that the indexer(s) in your instance receive, including Active Directory and Windows data. See "Install a license" in the Splunk App for Microsoft Exchange documentation. ... View more

I worked on this app quite a bit after my posts above and found that the correct index is now wineventlog, which changed from the previous version. When digging through the saved searches that the app uses, it references the index and sourcetypes correctly. There seems to be a bug with a few of the drop downs which I have seen in multiple environments across multiple companies and even after clean installs. I have worked with splunk on these dropdowns but they don't seem to have an answer at this point and I get the feeling that that original devs are no longer working on it. ... View more

Hi! What has worked for me, especially when I would run into permission issues early in my splunking career, is to follow the steps listed above, but then add the following touches (assuming splunk is the user you want to use): sudo $SPLUNK_HOME/bin/splunk stop (no need to have splunkd cling to files/process that retain the previous ownership) sudo su splunk sudo chown -R splunk:splunk /opt/splunk(or where ever splunk is installed) sudo $SPLUNK_HOME/bin/splunk start Let splunk run through it's initialization process and BAM! Splunk is running as the new user, all of the ownership should be changed recursively throughout the file structure, and you've removed the need to reindex data or run chown multiple times. Hopefully this works with the same magical flair for you as it has for me. But you have shout "Bam!" with an exaggerated motion or else you break the magic. 🙂 happy splunking, my friend. ... View more

Since this is an accepted answer, I have added it as a workaround. ... View more