We created a dashboard where $d_name$ in following search is user input:
<search> <query> <![CDATA[index=application host="landing.itsc.cuhk.edu.hk" POST OR GET status<400 $d_name$ | rex field=uri "\/(?<deptcode>[^\/]+)\/" | fields deptcode useragent| search deptcode=$d_name$ | timechart count ]]> </query> </search> <fieldForLabel>deptcode</fieldForLabel> <fieldForValue>deptcode</fieldForValue> </input>
Seems using post-process search is slower in our environment as there are more than 1 search peer. Is that correct?
Then I try to convert the inline searches into reports and thus accelerate wherever possible. However, can I do that with input field to be passed to the saved search/report, while base search is not used?
Thanks a lot.
I am assuming you have multiple Search Heads and Multiple Indexers. Splunk documentation mentions that Post-Processing is not useful in case of Multiple Indexers, it also suggests to reuse the same search in suce case (refer to Post-process searches) : http://docs.splunk.com/Documentation/Splunk/latest/Viz/Savedsearches
A post-process is effective for conserving resources if the base search is on a single indexer. If you are in an environment with a search head using multiple indexers, post process might not be effective for conserving resources. In this scenario, it might be more effective to use the same search multiple times within a dashboard.
as first time you could extract deptcode field before, in this way you can insert the search for this field in the primary search: this accelerate your search!
see http://docs.splunk.com/Documentation/Splunk/6.5.1/Knowledge/Aboutsummaryindexing and http://docs.splunk.com/Documentation/Splunk/6.5.1/Report/Acceleratereports
You could create a summary index (e.g. using tscollect command) and then use tstats command)
index=application host="landing.itsc.cuhk.edu.hk" POST OR GET status<400 | rex field=uri "\/(?<deptcode>[^\/]+)\/" | table _time deptcode useragent other_fields | tscollect namespace=mynamespace | tstats count AS cnt FROM mynamespace WHERE deptcode=$d_name$ GROUPBY _time deptcode useragent other_fields | timechart sum(cnt)
Thanks. Shall I put the tscollect statement as a base search in the dashboard, and use tstats in panel searches? Seems it's beneficial when there are lot of panels that need tstats as the tscollect command takes some time to build. The dropdown list built from search is the first one in the panel, and takes some time before user can get the list (tscollect + tstats).
Besides, chances are user will select the "All" option which is "deptcode=*". Seems tstats can't do with wildcard.
No you have to insert tscollect in a scheduled search and use tstats in your panels.
Tscollect creates a parallel index with less fields, all indexed and more quicker.
If making tscollect in a scheduled search, shall I specify time range based on the scheduled, say running everyday, with time range of -1d@d so that all data will be tscollect'd?
Yes, tscollect schedule is related to your needs about near real time monitoring: so you can schedule one time a day using -d@d - @d time period or less time.
Hi, I run tscollect multiple times for some testing. Seems data will be counted multiple times if I run tscollect on the same time range repeatedly. Is that true?
How can I delete the 'namespace' (or tsindx file) created by tscollect?
Thanks a lot.