Getting Data In
Highlighted

How to set up Indexes on Indexers

Motivator

HI

We have installed a SH and 4 INDEXERS(Non Clustered). We have installed our app to the SH only with our indexers=mlc_live and or datamodels.
We have set up the forwarders to send data to the INDEXERS, however the SH is giving us errors saying

"Search peer hp400srv6000INDEXER1 has the following message: Received event for unconfigured/disabled/deleted index=mlclive with source="source::/net/dell429srv/dell429srv1/apps/QCSTRSATv3.1.43SEC1/logs/traces/mxtiming286120dell429srv80849.log" host="host::NICKNAME" sourcetype="sourcetype::MXTIMING2".

So the INDEXERS dont know about the Index=MLC_LIVE, so 3 questions

Do i manually set up indexes on indexers?
How do i manage my APP on my SH, so changes get passed over to all indexers?
Should i use the Deployer to move changes I make to get pushed over to the INDEXERS, like datamodels changes etc...?

Cheers in advance
Rob

0 Karma
Highlighted

Re: How to set up Indexes on Indexers

Path Finder

Since your environment is not clustered, you will want to create the index on each indexer. You can do this via the UI or from the CLI. Look at the Wiki below:

Splunk Web:

  1. In Splunk Web, navigate to Settings > Indexes and click New.
  2. To create a new index, enter:
  3. A name for the index. User-defined index names must consist of only numbers, lowercase letters, underscores, and hyphens. They cannot begin with an underscore or hyphen, or contain the word "kvstore".
  4. The index data type. For event data, click Events. This is the default data type.
  5. The path locations for index data storage: Home path. Leave blank for default $SPLUNKDB/<indexname>/db Cold path. Leave blank for default $SPLUNKDB/<indexname>/colddb Thawed path. Leave blank for default $SPLUNKDB/<indexname>/thaweddb
  6. Enable/disable data integrity check.
  7. The maximum size of the entire index. Defaults to 500000MB.
  8. The maximum size of each index bucket. When setting the maximum size, use autohighvolume for high volume indexes (such as the main index); otherwise, use auto.
  9. The frozen archive path. Set this field if you want to archive frozen buckets. For information on bucket archiving, see Archive indexed data.
  10. The app in which the index resides.
  11. The tsidx retention policy. See Reduce tsidx usage. For more information on index settings, see Configure index storage.
  12. Click Save.

CLI:
Edit indexes.conf

To add a new index, add a stanza to indexes.conf in $SPLUNK_HOME/etc/system/local, identified by the name of the new index. For example:

[newindex]
homePath=
coldPath=
thawedPath=
...
For information on index settings, see Configure index storage and the indexes.conf spec file.

Note: User-defined index names must consist of only numbers, lowercase letters, underscores, and hyphens. They cannot begin with an underscore or hyphen, or contain the word "kvstore".

You must restart the indexer after editing indexes.conf.

https://docs.splunk.com/Documentation/Splunk/7.3.1/Indexer/Setupmultipleindexes

View solution in original post

0 Karma
Highlighted

Re: How to set up Indexes on Indexers

SplunkTrust
SplunkTrust

This is a good answer. I would add a strong recommendation to put your indexes.conf file into a custom app (myorgallindexes, for example) and install that app on all indexers. This helps avoid errors from making changes manually.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Highlighted

Re: How to set up Indexes on Indexers

Path Finder

Excellent point Rich! Since he isn't running a clustered environment, he could use a Deployment Server to deliver his custom app to the indexers and avoid having to move it to each.

0 Karma
Highlighted

Re: How to set up Indexes on Indexers

Motivator

Hi

It is looking like i will use the Deployment Server
I will give it a go and get back. Just to ask one more question.

Will the Deployment Server be able to push out real time updates, as we update the APP in production daily. So we need to push out updates to data-models specifically.

Thanks for the help to all

Rob

0 Karma
Highlighted

Re: How to set up Indexes on Indexers

Path Finder

Clients will check in periodically and compare the app on the DS to their app. If there is a change, the client will download.

https://docs.splunk.com/Documentation/Splunk/7.3.1/Updating/Howdeploymentupdateshappen

Process for setting up the DS:

https://docs.splunk.com/Documentation/Splunk/7.3.1/Updating/Planadeployment

0 Karma
Highlighted

Re: How to set up Indexes on Indexers

SplunkTrust
SplunkTrust

Data models are stored on search heads, not indexers. The data saved by DMs is stored on the indexers.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Highlighted

Re: How to set up Indexes on Indexers

Motivator

Hi

Thanks for your help

We are seeing that the DataModels are stored on the Indexers not the search heads.

/splunk/var/lib/splunk/mlclive/datamodelsummary

Cheers
Rob

0 Karma