topic Re: Streamlining with tstats in Splunk Search

Streamlining with tstats

supersnedz — Thu, 01 Feb 2024 15:08:34 GMT

Hi all,

im looking to create a dashboard to capture various info on or proxy data. I have a few simple queries

index=siem-proxy | top limit=5 cs_method

and my other query

index=siem-proxy | top limit=8 dest_port

this gets the requests methods such as POST, GET etc. I want to add this to a dashboard but looking to streamline the query first, i tried using tstats but was getting nothing back some I think im getting the syntax wrong. Without streamlining the queries are taking a very long time to run as i have millions of events. Is there a way to put this into a tstats query that I can use as visualization?

thank you

Re: Streamlining with tstats

richgalloway — Thu, 01 Feb 2024 15:11:24 GMT

What tstats commands have you tried so far?

A key thing to remember about tstats is that it can only search for index-time fields or fields in an accelerated datamodel. Use the walklex command to see if the fields you want to use are indexed.

Re: Streamlining with tstats

Richfez — Thu, 01 Feb 2024 15:16:38 GMT

tstats only works with index-time fields, and those fields are all likely to be search-time.

Alternatives in this case to try to improve performance:

1) Report acceleration, which should let you create a saved report that keeps most of the details of the search up-to-date, then you refer to that in your dashboard and it should drop the time *way* down.

2) Or build a data model (if one doesn't exist in the CIM add on already) for this data, and accelerate the data model. Similar to above in overall speed, but quite a bit different under the hood and more flexible.

Both of these (and some other options) are in the docs, well explained, here:

https://docs.splunk.com/Documentation/Splunk/9.1.3/Knowledge/Aboutsummaryindexing

I hope that gets you started!

Happy Splunking!

Re: Streamlining with tstats

supersnedz — Thu, 01 Feb 2024 15:25:35 GMT

Ah Thank you Richfez, i was unaware the tstats only worked with time related fields. I will have a read through that document you linked and work from there.

Thanks again really appreciate it

Re: Streamlining with tstats

Richfez — Thu, 01 Feb 2024 15:57:47 GMT

Well, tstats works with more than just time fields. The limitation is that it only works with fields that are created *at* the time the events are indexed.

https://docs.splunk.com/Documentation/Splunk/9.1.3/Indexer/Indextimeversussearchtime

I honestly think some of that information about performance or whatever is outdated, but most of that's all still fine documentation.

Index time fields are those created when the data's indexed. By default it's just the built-in fields, like _time, sourcetype, and so on.

In some cases it's all the fields, for instance with INDEXED_EXTRACTIONS=<json/csv/whatever>.

But otherwise Splunk generally relies on search time fields - fields that are built "on the fly" as you run your search. It's more flexible and doesn't go 'out of date' as events change or your needed fields change around.

The docs above should have links off them to explain more, but that's the gist of it.