topic Re: Heavy query in Monitoring Splunk

Heavy query

spillo491 — Wed, 12 Apr 2017 12:34:40 GMT

Hi,

I have a problem with this query executed on 3 months

index="app" host="sl0920*" source="/home/java/jboss-eap-6.2/app/log/teller-web.log" OR source="/home/java/jboss-eap-6.2/app/log/desktop-web.log" priority="FATAL" category="AUDIT*" message="{Invoking*" | eval date = strftime(_time, "%Y-%m-%d") | stats count as Contatore by message,correlationId,date | where Contatore > 1 | eval tot = Contatore/2 | chart sum(tot) as tot by date

Is it possible rewrite the same query with a query with better performances please ?

Re: Heavy query

DalJeanis — Wed, 12 Apr 2017 14:24:35 GMT

I don't see anything obviously bad about the code. When possible, you want to avoid reformatting data at the event level before the data is summarized, and a numeric field (or epoch time field) is more efficient for summary than a display field, but you at least need to bin the _time at the 1d level, so we can't avoid a reformat completely.

Try this, and see what happens. There might be a marginal improvement.

 index="app" host="sl0920*" (source="/home/java/jboss-eap-6.2/app/log/teller-web.log" OR source="/home/java/jboss-eap-6.2/app/log/desktop-web.log") priority="FATAL" category="AUDIT*" message="{Invoking*" 
| table _time, message, correlationId
| bin _time as date  span=1d
| stats count as Contatore by message, correlationId, date 
| where Contatore > 1 
| eval tot = Contatore/2
| eval date = strftime(_time, "%Y-%m-%d")  
| chart sum(tot) as tot by date

I'm tempted to try a timechart version, but without your data, I have no way of knowing if it would get better results.

Re: Heavy query

spillo491 — Wed, 12 Apr 2017 14:54:14 GMT

Sorry but this query doesn't extract any results

Re: Heavy query

niketn — Wed, 12 Apr 2017 15:31:26 GMT

@spillo491... do you get any result when you run 1st 4 lines of DalJeanis' code?

Re: Heavy query

niketn — Wed, 12 Apr 2017 15:32:39 GMT

On a different thought... If you have too many events to be handled in 30 days... You can actually try summary index.

Re: Heavy query

spillo491 — Thu, 13 Apr 2017 06:40:58 GMT

Yes, I get results !