Getting Data In
Highlighted

How to determine what causes the unevenness of the indexing rate across the indexers?

Motivator

The indexer with the highest indexing rate has a double indexing rate than the lowest indexer, based on the Monitoring Console.
How can see this rate for the indexers for a month or so? and hopefully with the hosts that each one gets data from.

Tags (2)
0 Karma
Highlighted

Re: How to determine what causes the unevenness of the indexing rate across the indexers?

SplunkTrust
SplunkTrust

there are plenty of things to check, below are the first steps i will suggest

start here, this will tell you how many events are indexed and how many unique hosts are sending data to each indexer
| tstats count as event_count dc(host) as u_host where index=* by splunk_server
if you have very uneven numbers there, start looking at outputs.conf and verify your hosts have the appropriate outputs.conf configurations
you can also start by checking load over time:
| tstats count as event_count where index=* by splunk_server _time span=1d | timechart span=1d max(event_count_ as total_events by splunk_server

hope it leads you in the right direction

View solution in original post

Highlighted

Re: How to determine what causes the unevenness of the indexing rate across the indexers?

Motivator

Thank you @adonio.

When running - | tstats count as event_count by splunk_server _time span=1d | timechart span=1d max(event_count) as total_events by splunk_server we see the following -

alt text

How come some of the cells are empty? These indexers were up and running every day...

0 Karma
Highlighted

Re: How to determine what causes the unevenness of the indexing rate across the indexers?

SplunkTrust
SplunkTrust

forgot to add the where clause to the tstats see my fixed above
also, if there is no data / events that day for that index, then its 0 / null

Highlighted

Re: How to determine what causes the unevenness of the indexing rate across the indexers?

Motivator

Really really interesting @adonio - adding the where clause changed everything.

0 Karma
Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.