Azure Servers not powerful enough

[Abass42]

I had a quick question about the resources on my indexer. I have a dev environment with a forwarder, indexer, and SH. On all of the servers, I have an IO Wait error. Investigating, I could turn that alert off, or I could look at the actual resources available on the machine. Looking through it, it looks as if i may need more resources. Looks like i only have 2 cores? and about7 GB of ram. 

Min Specs recommended by Splunk are:

• An x86 64-bit chip architecture.
• 12 physical CPU cores, or 24 vCPU at 2 GHz or greater per core.
• 12 GB RAM.

This is what i have:

Would this explain these errors:

System iowait reached red threshold of 3
Maximum per-cpu iowait reached red threshold of 10
Sum of 3 highest per-cpu iowaits reached red threshold of 15

Before I started trying to re do our Dev env from the ground up, we were receiving these errors and they haven't gone away. 

Thanks for any help

[isoutamo]

Hi

I said that for working dev environment you should have at least 4vCPU and 8GB memory. But even more important is that your disks can perform at least 800IOPS preferred is 1200+ IOPS. This should apply both Splunk binary/var and splunk indexer data disks.

One way to test this is use Bonnie++ or some similar tool. Of course if you see that information from your infra tools it's enough.

r. Ismo

[deepakc]

This indicates  that the CPU is spending a significant amount of time waiting for I/O  (typically disk) as your ingesting/parsing data/searching, so with Splunk you need to size it sufficiently, otherwise you will get those messages, remember Splunk is a workhorse and needs resources:

Have a look at the below to posts, I recently had replied to around iowait

https://community.splunk.com/t5/Splunk-Enterprise/IOWAIT-Mystery-What-is-it-Is-it-important/m-p/6902... 

https://community.splunk.com/t5/Splunk-Enterprise/Splunk-Enterprise-how-does-it-detect-IOWAIT-warnin... 

Go through these questions

https://docs.splunk.com/Documentation/Splunk/9.2.1/Capacity/Performancechecklist 

Look at the guide in terms of performance recommendations 

https://docs.splunk.com/Documentation/Splunk/9.2.1/Capacity/Summaryofperformancerecommendations

 In summary I think you will need to bump up your specifications, but for a dev environment, you can ignore those messages, unless it's starts to crawl and become unbearable.