topic Re: Splunk app data index issue in Getting Data In

Splunk app data index issue

yasit — Thu, 21 Sep 2023 13:11:23 GMT

my app contains the index.conf which declares the index that is installed on the heavy forwarder and it is not installed on the indexer. The problem is that data does not land on the indexer

Re: Splunk app data index issue

gcusello — Thu, 21 Sep 2023 13:23:03 GMT

Hi @yasit,

you have two choices:

install the app also on Indexers (I don't hint),
manually create the index on the Indexer.

usually this is described in the instructions, which is the app?

Ciao.

Giuseppe

Re: Splunk app data index issue

yasit — Thu, 21 Sep 2023 13:29:13 GMT

thanks @gcusello

what seems to be the issue? my understanding was that by default if Splunk receives data for an index that doesn't exist, it will attempt to create the index dynamically.

Re: Splunk app data index issue

dural_yyz — Thu, 21 Sep 2023 13:30:24 GMT

Agreed - you need to have the index defined on the indexers. Since the HF cooks the data when it comes across you need to have matching configuration at the receiving side. Failure to do this will mean your data will route to the last chance index.

On the indexer check btool config for indexes.conf

[default]
lastChanceIndex = <index name>
* An index that receives events that are otherwise not associated
  with a valid index.
* If you do not specify a valid index with this setting, such events are
  dropped entirely.
* Routes the following kinds of events to the specified index:
  * events with a non-existent index specified at an input layer, like an
    invalid "index" setting in inputs.conf
  * events with a non-existent index computed at index-time, like an invalid
    _MetaData:Index value set from a "FORMAT" setting in transforms.conf
* You must set 'lastChanceIndex' to an existing, enabled index.
  Splunk software cannot start otherwise.
* If set to "default", then the default index specified by the
  'defaultDatabase' setting is used as a last chance index.
* Default: empty string

Re: Splunk app data index issue

gcusello — Thu, 21 Sep 2023 13:33:42 GMT

Hi @yasit,

it isn't correct: if you are trying to send logs to a not existing index, you have a message (someting like this: "unconfigured/disabled/deleted index=wineventlog with source="source::WinEventLog:System"), but the index isn't automatically created.

Ciao.

Giuseppe

Re: Splunk app data index issue

yasit — Thu, 21 Sep 2023 13:37:13 GMT

@dural_yyz Thanks for the insight,
I've declared the index in my app's indexes.conf which is installed on the HF which essentially is being populated by scripted input.
But is there a way around where I don't have to install my app on the indexers? And also can you please provide the reference where it mentions that I have to install my app in Indexer?

Re: Splunk app data index issue

dural_yyz — Thu, 21 Sep 2023 13:41:25 GMT

https://docs.splunk.com/Documentation/Splunk/9.1.1/Indexer/Setupmultipleindexes

You don't have to add your app to the indexers but you must define your index on the indexers. A stand alone instance can define via GUI management, however if you have an indexing cluster you must use the CLI to edit an indexes.conf file which is pushed in the CM bundle to the IDX tier.