tstats summariesonly=t gets no results on accelerated data models

[responsys_cm]

I have installed the CIM app done all of the event typing and tagging to get my data into the data models relevant to my environment. I have accelerated those data models. It's a clustered environment with six indexers and a single search head.

If I run the tstats command with the summariesonly=t, I always get no results.

I've looked in the internal logs to see if there are any errors or warnings around acceleration or the name of the data model, but all I see are the successful searches that show the execution time and amount of events discovered.

Any ideas on how to troubleshoot this? I'd prefer for my dashboards to run only of the TSIDX data rather than raw events.

Thx.

[woodcock]

If it doesn't work with summariesonly=f then it never will with summariesonly=t. You have to ensure that you did your CIM-mapping correctly and it sounds like you did not.

[khowson]

You're missing the point. That's okay.

It does work with summariesonly=f. The issue is with summariesonly=true and the path the data is contained on the indexer.

This does not work:
| tstats summariesonly=true count from datamodel=Network_Traffic.All_Traffic where * by All_Traffic.src

however this does:
| tstats summariesonly=true count from datamodel=Network_Traffic where * by All_Traffic.src

[khowson]

Try removing part of the datamodel objects in the search. so try

| tstats summariesonly count from datamodel=Network_Traffic where * by All_Traffic.src

Instead of:

| tstats summariesonly count from datamodel=Network_Traffic.All_Traffic where * by All_Traffic.src

Let meknow if that work.

[kamlesh_vaghela]

Hi @responsys_cm,

You are not getting any data in tstats search with and without summariesonly, right?
Well I assume you did all configuration check from data model side So is it possible to validate event side configurations? Can you please check it by executing search from constraint in data model, with each level (if any). This will return error if any. If it is then it would be cause to make data model in building state(Not build state) . Verify the eventtype and tag by executing search if it is used in any constraint. I hope this will help you.

Thanks

[responsys_cm]

With summariesonly=t, I get nothing. Without summariesonly=t, I get results.

The problem seems to be that when the acceleration searches run, they find no results. And yet | datamodel XXXX search does.

[kamlesh_vaghela]

Without summariesonly=t you get results. Well means the datamodel don't have any accelerated data as @493669 mentioned. @responsys_cm can you please let us know the acceleration period and the volume of events for that particular period? As you mentioned above, your data-model is stuck in building state So might be this information helpful to us.

Thanks

[493669]

summariesonly=t flag, this tells the data model only to look at existing accelerated data (tsidx data) that has been automatically generated by the acceleration and non-summarized data will not be provided

[micahkemp]

Can you include your search strings for both the working and non-working searches?

[responsys_cm]

Here is what I see in the logs for the Change Analysis data model:

02-06-2018 17:12:17.529 +0000 INFO  SavedSplunker - savedsearch_id="nobody;Splunk_SA_CIM;_ACCELERATE_DM_Splunk_SA_CIM_Change_Analysis_ACCELERATE_", search_type="datamodel_acceleration", user="nobody", app="Splunk_SA_CIM", savedsearch_name="_ACCELERATE_DM_Splunk_SA_CIM_Change_Analysis_ACCELERATE_", priority=highest, status=success, digest_mode=1, scheduled_time=1517937120, window_time=0, dispatch_time=1517937120, run_time=0.492, result_count=0, alert_actions="", sid="scheduler__nobody_U3BsdW5rX1NBX0NJTQ__RMD554b34bdbf03a626a_at_1517937120_10599", suppressed=0, thread_id="AlertNotifierWorker-2"

It runs every 5 minutes, but the result count is always zero.

Yet | datamodel Change_Analysis search earliest=-5m@m returns several hundred results.

I have the backfill period set to 1 day and the timeout for the acceleration search is set to 86400 seconds. Parallelization is set to the default of 3.

This search works: | tstats prestats=t count(Malware_Attacks.signature) FROM datamodel="Malware" BY _time span=1d

This does not: | tstats summariesonly =t prestats=t count(Malware_Attacks.signature) FROM datamodel="Malware" BY _time span=1d

[micahkemp]

Have you checked the status of the acceleration?

Settings -> Data models -> Expand arrow next to the datamodel in question

Under "Acceleration" you should see statistics relevant to the acceleration of this specific datamodel, including percent complete.

[responsys_cm]

They're stuck at "Building" with zero disk space usage.

[micahkemp]

The datamodels haven't been summarized, likely due to not having matched events to summarize, so searching with summariesonly=true is expected to return zero results.

Have you tried searching the data without summariesonly=true or via datamodel <datamodel name> search to see if it seems like the datamodel actually finds any data?

[responsys_cm]

Using | datamodel bla bla search returns results. As does searching without summariesonly=true and tstats.

We enabled acceleration on these data models a while back, so something else is the issue...

[responsys_cm]

I think the issue is that the backfill value is too high and the searches are timing out before the initial acceleration.

[cmerriman]

you can search an datamodel without the acceleration built out, which is why the tstats and datamodel commands bring back data. what summary range are you accelerating? are you using Parallel summarization?

[responsys_cm]

For the high volume data sources (tens of millions of daily events), I've tried setting the backfill to only be a day. For the lower volume ones with periodic data, like vulnerability scans, those are set to a week.

But even the data models that have a relatively small amount of data and a modest backfill aren't accelerating...

[graju89]

@responsys_cm I am having the same issue. Any luck for you in fixing this?