Knowledge Management

how to retain _time in summary index

Path Finder

when I try simple below query its taking the current system time instead of _time of original event.

splunk version: 6.6.3

index=indexname | collect index=si

I want the events in the summary index to retain the _time as it is in the primary index. But it's storing the current system time.

0 Karma

Communicator

Hi all,

Actually, this is a very good question. The only thing is that is asked wrong ....

How, anyone like myself, could run collect command from an index to another, and retain the original _time of the older index????

Does anyone think of that?

Thank you,
Bogdan.

0 Karma

Path Finder

Thanks Bogdan and also for your time in trying it out.

Still waiting for answer..

0 Karma

Influencer

@AnilPujar - What is your use case and the collect will give you timestamp of your events in index your are searching on. In case there is no timestamp in your main search only then it takes current system time. You can run in testmode and compare the _time from your main index to the result after collect.

0 Karma

Path Finder

Try this:
index=indexname | eval _time = Timestamp field in event> | collect index=si

0 Karma

Path Finder

What is the _time precedence for collect command?

0 Karma

SplunkTrust
SplunkTrust

You're not understanding how a summary index works. You should stay away from this until you get more experience

0 Karma

Path Finder

In Splunk, for everything there is a precedence from conf files to _time

This is how splunk assign _time to the events when getting data in.

https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/HowSplunkextractstimestamps

so similarly due to some precedence i'm missing _time in summary index.

Might be my existing index data unable to recognize timestamp

@skoelpin "So if you don't know the answers stay away from giving answers and I can see your experience with answers made."

0 Karma

SplunkTrust
SplunkTrust

I'm not sure how to interpret this.. If you're unsure that your populating search may not be extracting _time correctly then you have much deeper problems than writing new data to a summary index. Your original search is just piping raw data from one index to another, completely subverting the purpose of a summary index.

0 Karma

SplunkTrust
SplunkTrust

I think you're missing the point of a summary index. In your example, you are pushing raw data to the summary index rather than summarized data. You should have a transformational command prior to your collect command. Once the data is transformed, it creates metrics which can be shipped to a summary index. If you use a timechart command then _time will be passed to the summary index

0 Karma

Path Finder

Can we move this answer as comment to the question, since it is not the answer to the question asked.

index=indexname | collect index=si

What is the _time precedence if I run this command. The same _time of index=indexname should retain?

0 Karma

SplunkTrust
SplunkTrust

Why would I move this as a comment? From a technical standpoint, you are wasting resources and time shipping raw data to a summary index and getting zero benefit of the acceleration a summary index provides. This answer needs visibility for future Splunkers who may be looking to do the same thing

To be clear, DO NOT DO THIS

0 Karma

SplunkTrust
SplunkTrust

Well does your events in primary index has timestamp? According to docs, If you use the collect command with a time range of All time and the events do not have timestamps, the current system time is used for the timestamps.

0 Karma

Path Finder

Hi Mayur,

The data was collected using servicenow addon and i'm not seeing any timestamp being captured additionally.

Only _time mapped with sys_updated_on

0 Karma

Contributor

Hi,
You can add _time in your base search before collecting it in a new index

0 Karma

Champion
0 Karma

SplunkTrust
SplunkTrust

This solution is useless if he's sending raw data to a summary index..

0 Karma