topic Re: Logs Not Being Indexed in Getting Data In

Logs Not Being Indexed

cuppma — Thu, 27 Feb 2014 14:17:26 GMT

I'm fairly new to Splunk and I can't figure out how to get Splunk to index my logs. I have configured my WebSense device to send logs to Splunk on UDP 6667 and I have configured Splunk to listen for logs on UDP 6667. I did a packet capture to make sure the logs were getting sent to the Splunk server. I have confirmed that they are getting sent to the server, but I cannot search through the logs. I believe the logs are not being properly indexed. Any ideas?

Re: Logs Not Being Indexed

emaccaferri — Thu, 27 Feb 2014 14:28:47 GMT

Did you have set the sourcetype and the index when you configured Splunk?
Did you edited your local copy of props.conf and trasform.conf?

Re: Logs Not Being Indexed

cuppma — Thu, 27 Feb 2014 14:33:01 GMT

I set the sourcetype and index in the inputs.conf file. What do I need to edit in the props.conf and transform.conf files?

Re: Logs Not Being Indexed

alacercogitatus — Thu, 27 Feb 2014 14:45:25 GMT

What is your search that isn't working?

It might be a matter of time. Some logs from devices send in UTC, but if you are in EST, they won't show up! Try adding "latest=+10h@h" to your search and see if that makes your logs show up.

Re: Logs Not Being Indexed

cuppma — Thu, 27 Feb 2014 15:44:07 GMT

I am doing the most basic search something like "index=websense" just to see if the logs are even being indexed into Splunk and I'm getting no results. I tried adding "latest=+10h@h" that didn't work.

Re: Logs Not Being Indexed

alacercogitatus — Thu, 27 Feb 2014 15:47:59 GMT

Try looking in index=main or index=default. or just do "index=*" and see if they show. Make sure that index=websense is part of your inputs configuration.

Re: Logs Not Being Indexed

cuppma — Thu, 27 Feb 2014 15:58:16 GMT

index=main returns a lot of logs from Perfmon:LocalNetwork that look like this:

02/27/2014 10:55:26.775
collection=LocalNetwork
object="Network Interface"
counter="Current Bandwidth"
instance="Broadcom BCM5709C NetXtreme II GigE [NDIS VBD Client] _6"
Value=1000000000

Is it possible that those are the logs I want but the correct information isn't being extracted?

From inputs.conf
[udp://:6667]
sourcetype = websense
index = websense
disabled = false

Re: Logs Not Being Indexed

alacercogitatus — Thu, 27 Feb 2014 16:03:11 GMT

There's your typo - you have an extra :

Should be: [udp://6667]

Re: Logs Not Being Indexed

cuppma — Thu, 27 Feb 2014 16:04:57 GMT

Yea, I tried it with and without that colon. Still wasn't working. I should be setting this in /etc/apps/websense/local correct?

Re: Logs Not Being Indexed

alacercogitatus — Thu, 27 Feb 2014 16:06:10 GMT

Has the index been defined in indexes.conf or via the GUI? What does the _internal index say?

Re: Logs Not Being Indexed

cuppma — Mon, 28 Sep 2020 15:59:53 GMT

_internal index returned no results, I tried to set an index via the GUI and got the following error: "Timed out while waiting for splunkd daemon to respond. Splunkd may be hung." So I went into the server and configured /etc/apps/websense/local/indexes.conf with the following:

[websense]
homePath = $SPLUNK_DB\websense\db
maxDataSize = auto_high_volume
thawedPath = $SPLUNK_DB\websense\thaweddb
coldPath = $SPLUNK_DB\websense\colddb

Does that look correct? For some reason the backslashes are getting removed when I click comment, but they are in the config.

Re: Logs Not Being Indexed

alacercogitatus — Thu, 27 Feb 2014 16:23:34 GMT

Looks good to me, restart Splunk for it to take effect.

Re: Logs Not Being Indexed

cuppma — Thu, 27 Feb 2014 16:34:40 GMT

Still no results.

Re: Logs Not Being Indexed

cuppma — Thu, 27 Feb 2014 17:12:59 GMT

This may be a stretch, but I know my instance of Splunk listens for traffic from forwarders on TCP 6667. Could this be interfering with the UDP traffic? I know you can use the same port concurrently with both UDP and TCP but would doing this be an issue in Splunk?

Re: Logs Not Being Indexed

alacercogitatus — Fri, 28 Feb 2014 13:07:50 GMT

They shouldn't interfere with each other. Check the physical size of the index (The FireBrigade App can help with this). If the physical size isn't changing, you may have a listening problem.

Re: Logs Not Being Indexed

cuppma — Fri, 28 Feb 2014 15:38:20 GMT

I'm getting this error when trying to use the Fire Brigade App "Unable to fetch REST endpoint uri="/services/data/indexes?count=0" from server=""." I also am getting a lot of N/A, No results found when trying to use this app. Do I need to do some initial setup with this app?

Re: Logs Not Being Indexed

mgonter — Tue, 29 Sep 2020 06:44:08 GMT

You're forgetting directory slashes here.

homePath = $SPLUNK_DB/websensedb.

Unless you have a variable set for $SPLUNK_DBwebsensedb defined. Have you looked in $SPLUNK_HOME/var/lib/splunk to see if the index is there?