Getting Data In

How to use custom index for Universal Forwarder

mehmettecer
Explorer

Both of my servers are Linux OS and I am using latest Splunk 4.2.

I can forward from UF to Splunk to index, but everything - all monitor stanzas - is going to main index.

I am trying to configure UF so /var/log/case1 monitor indexed under test_case1 index, already created on Splunk server and /var/log/case2 monitor indexed under test_case2 index, also already created on Splunk server

But when I try to execute following commands on splunkforwarder I get error

splunk add monitor /var/log/case1 -index test_case1 splunk add monitor /var/log/case2 -index test_case2

Error is In handler 'monitor': Parameter index: Index 'test_case1' does not exist. Please provide a valid index.

I searched through Answers and Documentation but I cannot seem to put together a solution for this.

Any help is greatly appreciated.

Tags (1)
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

I'd consider this a bug in the CLI, but your best bet is to just edit the inputs.conf file, find the monitor stanza, and add the index=test_case2 line to it, e.g.:

[monitor:///var/log/case1]
index = test_case2

The file is probably in $SPLUNK_HOME/etc/apps/search/local/, but you might have to look for it:

./splunk cmd btool --debug inputs list monitor:///var/log/case

or

find $SPLUNK_HOME/etc -name inputs.conf | xargs grep -l '^\[monitor:///var/log/case'

View solution in original post

mic
Splunk Employee
Splunk Employee

This is problematic in 4.3.2 as you mentioned, but in 4.3.3 and going forward, there is a parameter (check-index) that you can set to make this to happen without getting an error message complaining about the nonexistent index. By setting check-index to false, Universal Forwarder would not require the index to be there to begin with, but this is not the default behavior in 4.3.3.

For example:

./splunk add monitor /var/log/case1 -index test_case1 -check-index false

The default behavior is different depending on the version

  • 4.3.3 universal forwarder: default check-index is true, which means that it would always check whether the index exists
  • 4.3.4 universal forwarder: default check-index is true, which means that it would always check whether the index exists
  • 4.3.5 universal forwarder: default check-index is true, which means that it would always check whether the index exists
  • 5.0.2 universal forwarder: default check-index is false
0 Karma

jbsplunk
Splunk Employee
Splunk Employee

This issue has been reported to support and a Enhancement Request has been filed on the behavior. The CLI has been implemented consistently across all Splunk Components.

There has been a feature request filed to allow for a remote index command line argument, and/or a warning message to ensure the user understands a remote index must exist if Splunk does not have one that exists locally.

gkanapathy
Splunk Employee
Splunk Employee

I'd consider this a bug in the CLI, but your best bet is to just edit the inputs.conf file, find the monitor stanza, and add the index=test_case2 line to it, e.g.:

[monitor:///var/log/case1]
index = test_case2

The file is probably in $SPLUNK_HOME/etc/apps/search/local/, but you might have to look for it:

./splunk cmd btool --debug inputs list monitor:///var/log/case

or

find $SPLUNK_HOME/etc -name inputs.conf | xargs grep -l '^\[monitor:///var/log/case'

Ayn
Legend

There was a very similar question posted a couple of hours ago that has an excellent answer:

http://answers.splunk.com/questions/12923/indexes-and-universal-forwarder

mehmettecer
Explorer

Hi Ayn, I already saw that answer and tried proposed solution but I still don't see my log files indexed under custom index I created.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...