topic Re: Speed up search using the following single index. in Splunk Search

Speed up search using the following single index.

slgizmo — Tue, 29 Sep 2020 15:07:36 GMT

I am working on creation of a dash board that consists of the following search and it does function and return the information that I am needing. Further, I have this set for Month to date so I can see how many users per day are connecting to the system. However, the search is very slow and I would like to find a way to make this a little faster. Is there anything I can do to this search string?

index=xd sourcetype="xendesktop:7:session" |dedup UserName date_mday | stats count by date_mday

Re: Speed up search using the following single index.

martin_mueller — Tue, 01 Aug 2017 12:52:24 GMT

This is a prettier search string doing essentially the same thing:

index=xd sourcetype="xendesktop:7:session" | timechart span=1d dc(UserName)

However, the bulk of the work will be event retrieval and parsing - I guess, without seeing your job inspector there are only guesses.
To speeding that up your best approach is to filter more. Is there a "connection established" event that contains all you need? If so, search for just that.

Once all the regular speedup approaches are exhausted, this type of search suits data model acceleration very well.

Re: Speed up search using the following single index.

slgizmo — Tue, 01 Aug 2017 12:56:41 GMT

Martin,

This does speed it up a little bit and is more condensed than what I have so thank you for this.

Scott

Re: Speed up search using the following single index.

gcusello — Tue, 01 Aug 2017 13:06:27 GMT

Hi slgizmo,
did you explored Splunk acceleration (http://docs.splunk.com/Documentation/Hunk/6.4.7/Hunk/Workwithreportacceleration)?

I usually use tscollect:
- create a search with the fields you need index=xd sourcetype="xendesktop:7:session" | dedup UserName date_mday | table _time UserName
- add | tscollect namespace=<namespace>
- schedule this search
- create a search like the following | tstats count AS cnt FROM <namespace> GROUPBY UserName | timechart span=1d sum(cnt) by UserName

Bye.
Giuseppe

Re: Speed up search using the following single index.

slgizmo — Thu, 03 Aug 2017 14:03:05 GMT

Giuseppe,

So I am trying your solution and if I understand properly when I create the search and run it a new TSIDX is created and allows the follow-up search using | tstats . The following is what I have set

Saved Search to be ran once a day
index=xd sourcetype="xendesktop:7:session" | dedup UserName date_mday | table _time UserName | tscollect namespace=DailyUserUsage

This shows me 86 for the count and then if I run the following

| tstats count AS cnt FROM DailyUserUsage GROUPBY UserName | timechart span=1d sum(cnt) by UserName

I get nothing returned. Any idea why?

Gizmo

Re: Speed up search using the following single index.

gcusello — Thu, 03 Aug 2017 15:06:58 GMT

run | tstats count AS cnt FROM DailyUserUsage GROUPBY UserName and see if you have results:
if there are results add _time after GROUPBY,
if you haven't results see if the search with tscollect gives results without tscollect.
Bye.
Giuseppe

Re: Speed up search using the following single index.

slgizmo — Thu, 03 Aug 2017 15:44:06 GMT

Giuseppe,
If I run the string prior to the tscollect I am seeing the events and Statistics. However, if I try to run | tstats I receive the following "Error in 'tstats' command: this command must be the first command of a search.

Re: Speed up search using the following single index.

martin_mueller — Thu, 03 Aug 2017 15:47:42 GMT

I highly recommend data model acceleration in favour of tscollect.

Re: Speed up search using the following single index.

slgizmo — Thu, 03 Aug 2017 15:50:51 GMT

Martin,
As I am a newbie when it comes to this I am willing to try anything to learn more about it. Do you have a recommended link that I can read on how to set data modeling up?
Gizmo

Re: Speed up search using the following single index.

martin_mueller — Thu, 03 Aug 2017 15:56:50 GMT

Tutorial: http://docs.splunk.com/Documentation/Splunk/6.6.2/PivotTutorial/WelcometothePivotTutorial
Actual Docs: http://docs.splunk.com/Documentation/Splunk/6.6.2/Knowledge/Aboutdatamodels http://docs.splunk.com/Documentation/Splunk/6.6.2/Knowledge/Acceleratedatamodels

Re: Speed up search using the following single index.

slgizmo — Thu, 03 Aug 2017 15:58:12 GMT

Martin,

Thanks I am looking into this now.

Gizmo

Re: Speed up search using the following single index.

gcusello — Thu, 03 Aug 2017 16:32:53 GMT

If you're satisfied by this answer, please accept it.
Bye.
Giuseppe