Knowledge Management

How to set the timestamp when using the collect command?

regriffith
Path Finder

I am searching yesterday's data and trying to insert it into an index for reporting purposes. I need to take multiple indexed events with various date/time fields and override them with the current date/time for the summary index table. The following search is a very simplified version that illustrates the issue.

index=blah

| eval _time=now()
| collect index=test

When I do the search, it inserts yesterday's date/time into the summary index _time field. Is there any way to reassign this?

Splunk 6.6.3.

Labels (1)

highsplunker
Contributor

wow. my problem was this snippet works ONLY when i put "T" in the timeformat.

| eval _time=strptime(time2, "%Y-%m-%dT%H:%M:%S.%3N")
0 Karma

PickleRick
SplunkTrust
SplunkTrust

This is a thread from so long ago and is about a long forgotten version.

Nowadays collect is much more flexible, especially if you're using output_format=hec

0 Karma

adobrzeniecki
Path Finder

So i figured out a way to retain _time. Whatever you are bringing over into your summary index; source, sourcetype, fields of your choice....Create your own _raw field. In my instance I created _raw as below:
| eval _raw= _time. ":" .source
| table _raw ALL OTHER FIELDS YOU WANT
| collect index=SI

This will retain the _time value in your summary index. If this works for you please upvote this response!

0 Karma

mayurr98
Super Champion

hey
you can set the value of the _time field before the collect.

Example:

... | addinfo | eval _time=info_max_time | collect index=test

you can use any eval function to calculate the _time value.

ddrillic
Ultra Champion

Maybe | collect index=test addtime=true

0 Karma

aliuwkina
Engager

It's working! But you need to use addtime=false and Splunk taking first timestamp in your event.

0 Karma

micahkemp
Champion

The docs suggest this would use info_min_time first, if present (which it sounds like it would be), and is true by default anyway:

addtime
Syntax: addtime=<bool>
Description: Use this option to specify whether to prefix a time field on to each event. Some commands return results that do not have a _raw field, such as the stats, chart, timechart commands. If you specify addtime=false, the Splunk software uses its generic date detection against fields in whatever order they happen to be in the summary rows. If you specify addtime=true, the Splunk software uses the search time range info_min_time. This time range is added by the sistats) command or _time. Splunk software adds the time field based on the first field that it finds: info_min_time, _time, or now().
Default: true
0 Karma

micahkemp
Champion

When you pipe events to collect with the _raw field present, that field will be used as the only value that gets indexed, and _time will be parsed from that text.

If you want to set your own _time, I suggest including only the fields you want and need in your summary index, like this:

index=blah | table user http_uri | eval _time=now() | collect index=test

Edit:

you could remove the old timestamp from the lines and insert a new one, but I would consider that a poor choice, as you're altering what was previously evidentiary quality data

Or perhaps you could collect the event with _raw in a new field. Try this:

index=blah | eval orig_raw=_raw | eval _time=now() | table _time orig_raw | collect index=test
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...