Splunk installed locally on mac not indexing data to any index other than main

[shivanshsingh]

I have splunk 6 installed locally on macbook pro. I have created an index called "test". I have also added a "Data Input" location to a directory from which I want my log files to get indexed to test index, but the data is not getting indexed. However, the log files are getting indexed from another directory to "main" index.
Need help !!

[sbrant_splunk]

According to that file, it should definitely be going to the "test" index. If I understand correctly, the data is being indexed, it's just going to the default index (main) instead of "test"? Is that correct?

If that's the case, there may be a conflicting configuration. You can find out what the active configuration is by utilizing the btool command. As you've seen now, there is more than one inputs.conf, that is the case with other config files as well.

You can see the compilation of inputs.conf files from various apps, as well as system, by running this command:

$SPLUNK_HOME/bin/splunk cmd btool inputs list --debug

You can find additional info about the btool command here: Splunk btool

[shivanshsingh]

As you mentioned, I ran a search on index=*, using all time, and I don't see any data being indexed from the log file of that directory.
How can I check if the splunk has right access to be able to read those log files?
Also, when I ran btool, yes I did see all the configurations as per the above inputs.conf file, though, not in the same order. Does that matter?

[sbrant_splunk]

Also, when you ran btool, you said you didn't see a conflict but did you see your configuration as expected?

[sbrant_splunk]

Which app that the input.conf resides in is really just a matter of organization, since Splunk puts them all together anyway. It won't matter if it resides in the search app.

As for the data not being indexed, there are a few things to check, first would be to check make sure that the user Splunk is running under has permissions to read the file you're monitoring. Second is, it's possible that the data is being indexed but not being timestamped properly. If you run a search on index=*, using real-time/all time, you'll see the data being indexed, regardless of the date/time.

[shivanshsingh]

No, the data is not being indexed at all. I searched with index=* and I don't see any logs coming from the directory mentioned in the inputs.conf file. This inputs.conf file is under Search App.

Though there is another directory which I have added earlier (this inputs.conf is under Launcher app) and configured to send that directory's log files to default index, and that is getting indexed correctly.

I did run the btool command and I don't see any conflicting entry for that directory.

PS: Does it matter under which app the inputs.conf is being configured or index is being created?

[shivanshsingh]

Thanks sbrant for pointing me to the inputs.conf file. Here is how the inputs.conf file looks. It's under "search" app. "/Applications/Splunk/etc/apps/search/local"

inputs.conf :

[monitor:///Users/ssingh9/Documents/Intuit/Logs/*.log]
disabled = false
followTail = 0
index = test
sourcetype = log4j

[sbrant_splunk]

OK, since you added the inputs via the web interface, they will be stored within the context of the app that you were in when you configured them. To find out what that is, you can go into settings > data inputs > files & directories, find your input and look to the right to see what app the configuration is in.

You will find that input.conf in $SPLUNK_HOME/etc/apps//local

[shivanshsingh]

Here is how my $Splunk_Home/etc/systems/local/inputs.conf file looks like:

===========
[default]
host = MTVL11b176e97.local

[monitor://$SPLUNK_HOME/var/log/splunk]

disabled = 1

Just to mention, the way I configured all the log files which needs to be indexed from one of my local directory is through, Settings> Data Inputs> Files and Directories. That's why those entries are not there in local/inputs.conf file.

[sbrant_splunk]

Can you provide the inputs.conf file where your inputs are specified?