Does tstats always specify a datamodel?

[Justin1224]

Basically my problem is that I'm switching Splunk queries that I have into queries for a different search language. I don't yet have the capability to transfer the part of the search that specifies where to search, be it datamodel=, or a count by: IDS_Attacks.severity (grouping by the field severity within the parent node IDS_Attacks). So my question is: is datamodel= part of every search?

Side question, does anyone who has experience with Elasticsearch know if you can/how to transfer these datamodel specifications to Elasticsearch query language.

[ddrillic · Accepted solution]

The following documentation tstats

says -

-- Use the tstats command to perform statistical queries on indexed fields in tsidx files. The indexed fields can be from normal index data, tscollect data, or accelerated data models.

For example, you can run the following on a normal index -

| tstats count where index=<index name> by _time, index, sourcetype span=1d

[Justin1224]

Why do you have to specify certain sets of data every time? What if you want to search in all of your data? Do you have to specify a certain subset of data every time you search?

[ddrillic]

It seems to be a versatile command. The examples are good -

| tstats count FROM mydata
| tstats avg(foo) FROM mydata WHERE bar=value2 baz>5
| tstats count WHERE host=x BY source
| tstats prestats=t count BY _time span=1d | timechart span=1d count
| tstats prestats=t median(foo) FROM mydata | tstats prestats=t append=t median(bar) FROM otherdata | stats median(foo) median(bar)

[Justin1224]

Right, what I'm unsure about though is: Is tstats .....blah...... datamodel=some_blah specifying a certain subset of your data to look through or something else? Like, when you have datamodel=some_blah in your search, is that telling you to only look inside "some_blah" to perform the search? And is that datamodel that you specify preexisting or do you create it?

[rjthibod]

Yes, it is restricting the search to that datamodel object, and that datamodel object must exist already. Datamodel's tsidx files are auto-generated by the datamodel acceleration subsystem. Ultimately, it is how tstats searches against these second-level indexes that gives you all the performance gains that come with accelerated datamodels.

[Justin1224]

A note: what exactly does datamodel = "name" do? I was under the impression that it specified a location: "name", inside all of your available data to perform your search. But I'm not sure that's what it does now. Can anyone give me clarification on this?

Thanks!