do not show in report if one column = 0

[jpburnison]

Hello my friends,

I'm new to splunk and still trying to figure out the ins and outs of everything. I have a report that's been handed down to me that I'm trying to improve on it's timing. The report runs at 3 AM EST every day, but sends out at 8 PM that night. I suspect this has to do with the amount of data the report is trying to collect and then display. To fix that, I want to trim down how much data we are building, but I don't know if it will actually help.

What we are currently showing is how many times each of our 320 dispensers are dispensing, how many of each error they are getting, the error rate, and their total # of errors. What I am thinking of doing is focusing on the total # of errors. Specifically, if a dispenser returns 0 errors, it's not in the report. Currently, the TotalErrors column is evaluated like this :

| eval TotalError=(Error1+Error2+Error3)

My thoughts are to evaluate against TotalError and if the # of errors = 0, not to display that dispenser.

[richgalloway]

Removing dispensers with no errors should help, but how much it helps depends on how much work is done beforehand.  If the filtering is late in the query then there is less benefit.

For maximum benefit, filter as early as possible.  For example,

index=foo (Error1>0 OR Error2>0 OR Error3>0)

Another way to improve performance is to be as specific as possible in the base query.  Always include the index name, but also include the sourcetype(s), host, and source where possible.  Avoid wildcards, especially leading wildcards.

Use the fields command to remove any fields you don't need.  This will reduce the amount of data Splunk is moving around and help speed things up.

Use centralized-distributable commands as late in the query as possible.  A centralized-distributable command sends the query back to the SH where it is processed by a single CPU rather than by multiple CPUs (one on each indexer).  A good example is table vs fields.  These commands are similar, but table is not distributable whereas fields is.

Consult the Job Inspector to see where the query is spending its time.  Focus on the most expensive operations first.