topic How to set the timestamp when using the collect command? in Knowledge Management

How to set the timestamp when using the collect command?

regriffith — Tue, 03 Sep 2024 18:06:20 GMT

I am searching yesterday's data and trying to insert it into an index for reporting purposes. I need to take multiple indexed events with various date/time fields and override them with the current date/time for the summary index table. The following search is a very simplified version that illustrates the issue.

index=blah | eval _time=now() | collect index=test

When I do the search, it inserts yesterday's date/time into the summary index _time field. Is there any way to reassign this?

Splunk 6.6.3.

Re: Setting the timestamp when using the collect command

micahkemp — Tue, 09 Jan 2018 22:28:40 GMT

When you pipe events to collect with the _raw field present, that field will be used as the only value that gets indexed, and _time will be parsed from that text.

If you want to set your own _time, I suggest including only the fields you want and need in your summary index, like this:

index=blah | table user http_uri | eval _time=now() | collect index=test

Edit:

you could remove the old timestamp from the lines and insert a new one, but I would consider that a poor choice, as you're altering what was previously evidentiary quality data

Or perhaps you could collect the event with _raw in a new field. Try this:

index=blah | eval orig_raw=_raw | eval _time=now() | table _time orig_raw | collect index=test

Re: Setting the timestamp when using the collect command

ddrillic — Tue, 09 Jan 2018 23:06:48 GMT

Maybe | collect index=test addtime=true

Re: Setting the timestamp when using the collect command

micahkemp — Tue, 09 Jan 2018 23:21:40 GMT

The docs suggest this would use info_min_time first, if present (which it sounds like it would be), and is true by default anyway:

addtime
Syntax: addtime=<bool>
Description: Use this option to specify whether to prefix a time field on to each event. Some commands return results that do not have a _raw field, such as the stats, chart, timechart commands. If you specify addtime=false, the Splunk software uses its generic date detection against fields in whatever order they happen to be in the summary rows. If you specify addtime=true, the Splunk software uses the search time range info_min_time. This time range is added by the sistats) command or _time. Splunk software adds the time field based on the first field that it finds: info_min_time, _time, or now().
Default: true

Re: Setting the timestamp when using the collect command

mayurr98 — Wed, 10 Jan 2018 05:22:15 GMT

hey
you can set the value of the _time field before the collect.

Example:

... | addinfo | eval _time=info_max_time | collect index=test

you can use any eval function to calculate the _time value.

Re: Setting the timestamp when using the collect command

adobrzeniecki — Thu, 02 Dec 2021 18:33:07 GMT

So i figured out a way to retain _time. Whatever you are bringing over into your summary index; source, sourcetype, fields of your choice....Create your own _raw field. In my instance I created _raw as below:
| eval _raw= _time. ":" .source
| table _raw ALL OTHER FIELDS YOU WANT
| collect index=SI

This will retain the _time value in your summary index. If this works for you please upvote this response!

Re: Setting the timestamp when using the collect command

aliuwkina — Fri, 29 Jul 2022 09:40:57 GMT

It's working! But you need to use addtime=false and Splunk taking first timestamp in your event.

Re: How to set the timestamp when using the collect command?

highsplunker — Tue, 03 Sep 2024 15:52:41 GMT

wow. my problem was this snippet works ONLY when i put "T" in the timeformat.

| eval _time=strptime(time2, "%Y-%m-%dT%H:%M:%S.%3N")

Re: How to set the timestamp when using the collect command?

PickleRick — Tue, 03 Sep 2024 17:48:24 GMT

This is a thread from so long ago and is about a long forgotten version.

Nowadays collect is much more flexible, especially if you're using output_format=hec