How do timestamps work with summary indexing?

[christoffertoft]

Im just now learning about summary indexing and have set up a search to run every hour, putting the results in a specific summary index. When I run the saved search in the splunk search bar, I get the unique timestamps for each event, however, if I search the index using the following search string:

index=mysummaryindex report=myreport 

The timestamps are all from 8:00:00.000 AM , 9:00:00.000 AM, etc. It seems like the timestamp is from the time when the saved search is ran. Is this expected behaviour? I was hoping to be able to see the original timestamps like when I run the search manually?

Regards, and thanks.

[DalJeanis · Accepted solution]

I marked your code as code, but it looks like some of the rex code is still missing. If you are wanting to collect each of the messages as individual messages, then you need to move toward the collect command, probably with a table in front of it to reduce the amount of data. On the other hand, if you want summary data -- say, a count of occurrences of each Message with the first and last _time in each hour -- then you need to build the data that you want to keep into the search before sending it to the summary index (also probably with collect, given your leanings).

[christoffertoft]

@DalJeanis thanks. the rex wasn't really important for the question but I thank you for the fix 🙂

I think we'll be moving towards Splunk ES and accelerated data models for the project instead of using summary indexing

[cpetterborg]

Since it is a summary table, what timestamp do you want? The one from the first event summarized, the average of the times, the median of the times, or that last event's timestamp? The reason for the timestamp as it is is to show the time of the beginning of the data that could be in the data. That is so the data can be searched to be within the timeframe the data is supposed to be summarizing. If you want a specific timestamp for the summary data, put a field in the summary data that is a timestamp that you want it to have. That way you can look at a timestamp for the summarized data the way you want it to be. Otherwise the timestamp of the summary event will be the beginning of the timeframe for the data within the summary event.

[christoffertoft]

I want the time stamp when it was first summarized. I want the data to look "exactly" like as if it was gathered from the original index.

[cpetterborg]

Are you summarizing each and every event as another event? If so, look at the collect command documentation where it tells you how to copy data to another index (usually a summary index) instead of using the summary indexing scheduled search.

If not, then I fail to understand how you can summarize data and have it look exactly like the original.

[adonio]

@christoffertoft,
how theresults from your search looks like?
will recommend to follow @cpetterborg last comment and use the | collect command for more summary indexing options

[jackreeves]

I am running a Summary Index based on a lookup table. However I want to change the summary index timestamp to "Month/Year" field (see below).

| inputlookup inventory.csv
| stats values(count) as count by Month/Year Month FamilyCod ProductCod

Month/Year field is in following format "%Y/%m". Could anyone advise?

[adonio]

can you share the search you are running?
are you using the | collect command ? or did you enable summary indexes from GUI?
read here in detail all the way through to learn more (3 long pages):
http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Usesummaryindexing?r=searchtip

[christoffertoft]

@adonio Any suggestions?

[christoffertoft]

the search im running is (masked):

 starthoursago=2 endhoursago=1 [|inputlookup servers | search host=*nt* | eval host=host+"*"] source=mySource rex field=Message ""

the summary-index check is enabled in the GUI (Settings -> Searches -> the search).

The added field is report=myreport.

When I try to extract the information in a search (to be used in a dashboard) i do index=myindex report=myreport. It shows the wrong timestamps.

[alemarzu]

Hi there, check this out.

http://docs.splunk.com/Documentation/Splunk/6.6.2/Knowledge/Usesummaryindexing#Summary_indexing_of_d...

[christoffertoft]

Like I said, the _time values exist in the original search.. If i do |eval _time=now() All the values will be preformatted to indexing time and not the original timestamp..