topic Re: Linux add disk space for Splunk indexes and logs in Installation

Linux add disk space for Splunk indexes and logs

jwilliams — Fri, 26 Jun 2020 02:55:23 GMT

Splunk Installation - I would like to know how to add disk space to splunk for indexes and logs. Where would this be in the fine manual?

Re: Linux add disk space for Splunk indexes and logs

gcusello — Fri, 26 Jun 2020 06:39:40 GMT

Hi @jwilliams ,

this isn't a Splunk problema: Splunk uses the disk space you give it.

To solve this problem you have to find in a Linux manual "lvextend" and it depends on the hardware you're using (virtual or physical.

Another solution to you problem could be adding another disk, so you can divide (in Splunk) indexes location in the two disks (using indexes.conf) or put the hot and warm buckets in the faster disk and cold buckets in the slower one (always using indexes.conf).

Ciao.

Giuseppe

Re: Linux add disk space for Splunk indexes and logs

isoutamo — Fri, 26 Jun 2020 08:44:29 GMT

Couple of best practices which we are using:

Always use LVM and volume groups on linux => You can extend space easily later one
- Add disk -> add it to vg -> extend current LV -> extend FS over that LV -> update indexes.conf to know bigger volume (if you are using volumes)
xfs is your friends (at least in older linux versions we have had some issues with ext4)
On splunk side use always volumes (after that your life is easier)
As @gcusello said, use separate tiers for hot/warm and cold data
If you are using Smartstore then there are some other guidelines to follow, partially little bit different than using only local storage
Always ensure that your disks are fast enough!
For high volume indexes use maxDataSize = auto_high_volume
Add enough earlier new indexers not just additional disk space to current indexers.
Usually it's good to user clusters as earlier as possible (even one node clusters for future expansions).

r. Ismo