Linux add disk space for Splunk indexes and logs

[jwilliams]

Splunk Installation - I would like to know how to add disk space to splunk for indexes and logs.  Where would this be in the fine manual?

[gcusello]

Hi @jwilliams ,

this isn't a Splunk problema: Splunk uses the disk space you give it.

To solve this problem you have to find in a Linux manual "lvextend" and it depends on the hardware you're using (virtual or physical.

Another solution to you problem could be adding another disk, so you can divide (in Splunk) indexes location in the two disks (using indexes.conf) or put the hot and warm buckets in the faster disk and cold buckets in the slower one (always using indexes.conf).

Ciao.

Giuseppe

[isoutamo · Accepted solution]

Couple of best practices which we are using:

• Always use LVM and volume groups on linux => You can extend space easily later one
  • Add disk -> add it to vg -> extend current LV -> extend FS over that LV -> update indexes.conf to know bigger volume (if you are using volumes)
• xfs is your friends (at least in older linux versions we have had some issues with ext4)
• On splunk side use always volumes (after that your life is easier)
• As @gcusello said, use separate tiers for hot/warm and cold data
• If you are using Smartstore then there are some other guidelines to follow, partially little bit different than using only local storage
• Always ensure that your disks are fast enough!
• For high volume indexes use maxDataSize = auto_high_volume
• Add enough earlier new indexers not just additional disk space to current indexers.
• Usually it's good to user clusters as earlier as possible (even one node clusters for future expansions).

r. Ismo