Getting Data In

How to use custom index for Universal Forwarder

mehmettecer
Explorer

Both of my servers are Linux OS and I am using latest Splunk 4.2.

I can forward from UF to Splunk to index, but everything - all monitor stanzas - is going to main index.

I am trying to configure UF so /var/log/case1 monitor indexed under test_case1 index, already created on Splunk server and /var/log/case2 monitor indexed under test_case2 index, also already created on Splunk server

But when I try to execute following commands on splunkforwarder I get error

splunk add monitor /var/log/case1 -index test_case1 splunk add monitor /var/log/case2 -index test_case2

Error is In handler 'monitor': Parameter index: Index 'test_case1' does not exist. Please provide a valid index.

I searched through Answers and Documentation but I cannot seem to put together a solution for this.

Any help is greatly appreciated.

Tags (1)
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

I'd consider this a bug in the CLI, but your best bet is to just edit the inputs.conf file, find the monitor stanza, and add the index=test_case2 line to it, e.g.:

[monitor:///var/log/case1]
index = test_case2

The file is probably in $SPLUNK_HOME/etc/apps/search/local/, but you might have to look for it:

./splunk cmd btool --debug inputs list monitor:///var/log/case

or

find $SPLUNK_HOME/etc -name inputs.conf | xargs grep -l '^\[monitor:///var/log/case'

View solution in original post

mic
Splunk Employee
Splunk Employee

This is problematic in 4.3.2 as you mentioned, but in 4.3.3 and going forward, there is a parameter (check-index) that you can set to make this to happen without getting an error message complaining about the nonexistent index. By setting check-index to false, Universal Forwarder would not require the index to be there to begin with, but this is not the default behavior in 4.3.3.

For example:

./splunk add monitor /var/log/case1 -index test_case1 -check-index false

The default behavior is different depending on the version

  • 4.3.3 universal forwarder: default check-index is true, which means that it would always check whether the index exists
  • 4.3.4 universal forwarder: default check-index is true, which means that it would always check whether the index exists
  • 4.3.5 universal forwarder: default check-index is true, which means that it would always check whether the index exists
  • 5.0.2 universal forwarder: default check-index is false
0 Karma

jbsplunk
Splunk Employee
Splunk Employee

This issue has been reported to support and a Enhancement Request has been filed on the behavior. The CLI has been implemented consistently across all Splunk Components.

There has been a feature request filed to allow for a remote index command line argument, and/or a warning message to ensure the user understands a remote index must exist if Splunk does not have one that exists locally.

gkanapathy
Splunk Employee
Splunk Employee

I'd consider this a bug in the CLI, but your best bet is to just edit the inputs.conf file, find the monitor stanza, and add the index=test_case2 line to it, e.g.:

[monitor:///var/log/case1]
index = test_case2

The file is probably in $SPLUNK_HOME/etc/apps/search/local/, but you might have to look for it:

./splunk cmd btool --debug inputs list monitor:///var/log/case

or

find $SPLUNK_HOME/etc -name inputs.conf | xargs grep -l '^\[monitor:///var/log/case'

Ayn
Legend

There was a very similar question posted a couple of hours ago that has an excellent answer:

http://answers.splunk.com/questions/12923/indexes-and-universal-forwarder

mehmettecer
Explorer

Hi Ayn, I already saw that answer and tried proposed solution but I still don't see my log files indexed under custom index I created.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...