Knowledge Management

Summary indexing and timechart: getting the wrong results but I don't know why

yoho
Contributor

Hi,

I would like to give access to indexing volume per day and per index to all my users but they must only be able to access to their own index + summarized index (to keep it simple, let's say userA has access only to indexA and summindexA).

So I would like to do this by scheduling several summary searches (1 per user) taking data out of the _internal index and writing the results to users' summary indexes. A second search will take care of displaying the results.

My saved search is the following:

[Index thruput A]
search =  index=_internal source=*metrics.log group=per_index_thruput series = indexA |     sitimechart span=1d sum(kb) by series
cron_schedule = 15 * * * *
action.summary_index = 1
enableSched = 1
dispatch.earliest_time = -1h@h
dispatch.latest_time = now@h
action.summary_index.name = summindexA

I copy/paste this stanza and just replace "A" by "B" for userB, C for userC, D for userD, etc...

The search displaying the results is:

[Log volume (last 30 days) A]
search = index=summindexA source="Index thruput A" | timechart span=1d sum(kb) as total_kb
dispatch.earliest_time = -30d
dispatch.latest_time = now

I get some results but they are wrong: for some reason, the first search appears to ignore dispatch.earliest_time and is not only summing results from the past hour but from the whole day.

Also I've noticed info_max_time is not present in the summary search results, meaning that checks for overlaps can probably not happen. Is it normal behavior of sitimechart ? (another sitimechart search appears to work the same way).

Update My syntax for the latest_time was wrong: "now@h" was invalid and I should have used "@h". Not sure this is the root cause of all the problems but it certainly didn't help to get the results right. I've updated the saved search and will investigate again if I now get more accurate results.

0 Karma
1 Solution

yoho
Contributor

Now that I've fixed the problem mentioned in my update I get info_max_time properly set. It should allow Splunk to check if there are any overlaps or gaps and only compute the sum for data which has not been taken into account yet.

So this should fix the problem and now work but in the end I found it simpler to set span=1h instead of span=1d and I finally got the right results.

View solution in original post

0 Karma

yoho
Contributor

Another reason is that I'm a bit new to summarized indexing and I've understood you have to put the same args to your "sitimechart" and "timechart" commands to have statistically correct results. I've indeed finally came down to an hourly span and get the correct results (see below).

0 Karma

yoho
Contributor

Now that I've fixed the problem mentioned in my update I get info_max_time properly set. It should allow Splunk to check if there are any overlaps or gaps and only compute the sum for data which has not been taken into account yet.

So this should fix the problem and now work but in the end I found it simpler to set span=1h instead of span=1d and I finally got the right results.

0 Karma

yoho
Contributor

Yes, that's something I've not explained. The reason is that I want to get the daily indexing volume over the past 30 days but for today I wouldn't like to wait until tomorrow before getting the results.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Any specific reason to use span=1d in a search which is running for last 60 minute only?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...