Knowledge Management

Setting the timestamp when using the collect command

regriffith
Path Finder

I am searching yesterday's data and trying to insert it into an index for reporting purposes. I need to take multiple indexed events with various date/time fields and override them with the current date/time for the summary index table. The following search is a very simplified version that illustrates the issue.

index=blah

| eval _time=now()
| collect index=test

When I do the search, it inserts yesterday's date/time into the summary index _time field. Is there any way to reassign this?

Splunk 6.6.3.

adobrzeniecki
Path Finder

So i figured out a way to retain _time. Whatever you are bringing over into your summary index; source, sourcetype, fields of your choice....Create your own _raw field. In my instance I created _raw as below:
| eval _raw= _time. ":" .source
| table _raw ALL OTHER FIELDS YOU WANT
| collect index=SI

This will retain the _time value in your summary index. If this works for you please upvote this response!

0 Karma

mayurr98
Super Champion

hey
you can set the value of the _time field before the collect.

Example:

... | addinfo | eval _time=info_max_time | collect index=test

you can use any eval function to calculate the _time value.

0 Karma

ddrillic
Ultra Champion

Maybe | collect index=test addtime=true

0 Karma

micahkemp
Champion

The docs suggest this would use info_min_time first, if present (which it sounds like it would be), and is true by default anyway:

addtime
Syntax: addtime=<bool>
Description: Use this option to specify whether to prefix a time field on to each event. Some commands return results that do not have a _raw field, such as the stats, chart, timechart commands. If you specify addtime=false, the Splunk software uses its generic date detection against fields in whatever order they happen to be in the summary rows. If you specify addtime=true, the Splunk software uses the search time range info_min_time. This time range is added by the sistats) command or _time. Splunk software adds the time field based on the first field that it finds: info_min_time, _time, or now().
Default: true
0 Karma

micahkemp
Champion

When you pipe events to collect with the _raw field present, that field will be used as the only value that gets indexed, and _time will be parsed from that text.

If you want to set your own _time, I suggest including only the fields you want and need in your summary index, like this:

index=blah | table user http_uri | eval _time=now() | collect index=test

Edit:

you could remove the old timestamp from the lines and insert a new one, but I would consider that a poor choice, as you're altering what was previously evidentiary quality data

Or perhaps you could collect the event with _raw in a new field. Try this:

index=blah | eval orig_raw=_raw | eval _time=now() | table _time orig_raw | collect index=test
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!