I'm fairly new to Splunk and I can't figure out how to get Splunk to index my logs. I have configured my WebSense device to send logs to Splunk on UDP 6667 and I have configured Splunk to listen for logs on UDP 6667. I did a packet capture to make sure the logs were getting sent to the Splunk server. I have confirmed that they are getting sent to the server, but I cannot search through the logs. I believe the logs are not being properly indexed. Any ideas?
What is your search that isn't working?
It might be a matter of time. Some logs from devices send in UTC, but if you are in EST, they won't show up! Try adding "latest=+10h@h" to your search and see if that makes your logs show up.
I'm getting this error when trying to use the Fire Brigade App "Unable to fetch REST endpoint uri="/services/data/indexes?count=0" from server="
They shouldn't interfere with each other. Check the physical size of the index (The FireBrigade App can help with this). If the physical size isn't changing, you may have a listening problem.
This may be a stretch, but I know my instance of Splunk listens for traffic from forwarders on TCP 6667. Could this be interfering with the UDP traffic? I know you can use the same port concurrently with both UDP and TCP but would doing this be an issue in Splunk?
_internal index returned no results, I tried to set an index via the GUI and got the following error: "Timed out while waiting for splunkd daemon to respond. Splunkd may be hung." So I went into the server and configured /etc/apps/websense/local/indexes.conf with the following:
homePath = $SPLUNK_DB\websense\db
maxDataSize = auto_high_volume
thawedPath = $SPLUNK_DB\websense\thaweddb
coldPath = $SPLUNK_DB\websense\colddb
Does that look correct? For some reason the backslashes are getting removed when I click comment, but they are in the config.
You're forgetting directory slashes here.
homePath = $SPLUNK_DB/websensedb.
Unless you have a variable set for $SPLUNK_DBwebsensedb defined. Have you looked in $SPLUNK_HOME/var/lib/splunk to see if the index is there?
index=main returns a lot of logs from Perfmon:LocalNetwork that look like this:
instance="Broadcom BCM5709C NetXtreme II GigE [NDIS VBD Client] _6"
Is it possible that those are the logs I want but the correct information isn't being extracted?
sourcetype = websense
index = websense
disabled = false
I am doing the most basic search something like "index=websense" just to see if the logs are even being indexed into Splunk and I'm getting no results. I tried adding "latest=+10h@h" that didn't work.