Getting Data In

Can the formatting of the timestamp added by the collect command be modified?

laserval
Communicator

I'm using collect to create a summary index of a subset of events that all have a specific field, with all field extractions done.
It basically looks like this:

index=coolstuffs where NOT isnull(piece_id) | eval rawevent=_raw | fields - _raw | collect addtime=true index=coolstuffs_piece_events`

This gives me events of the format:

11/14/2013 13:08:54 +0100, info_min_time=12324545.000, ..., some_extracted_field="value", rawevent="2013-11-14 14:08:54.067 INFO here is the raw event", piece_id="ASde123"

Which means the summary index has dropped sub-second accuracy, which breaks the ordering of the events when searching on the index. If I set addtime=false, no timestamp is added to the event (of course), which breaks the index even more as splunk can't find the timestamp because I removed _raw (necessary to get the field extractions).

Is there any way of configuring the format for the timestamp collect uses?

I'm using Splunk 6 - on Windows.

0 Karma
1 Solution

laserval
Communicator

And now I think I solved this. Apparently collect takes the timeformat option that search also has - which can change the format of the output timestamp.

I'm now doing this:

| collect timeformat="%F %T.%3Q %z" addtime=true index=...

This gives me events that look like this:

2013-11-14 13:54:55.992 +0100, search_name=...

Much more usable. I don't really see the reason for dropping sub-seconds in the default timeformat. It also seems like an undocumented behaviour.

View solution in original post

laserval
Communicator

And now I think I solved this. Apparently collect takes the timeformat option that search also has - which can change the format of the output timestamp.

I'm now doing this:

| collect timeformat="%F %T.%3Q %z" addtime=true index=...

This gives me events that look like this:

2013-11-14 13:54:55.992 +0100, search_name=...

Much more usable. I don't really see the reason for dropping sub-seconds in the default timeformat. It also seems like an undocumented behaviour.

jtrucks
Splunk Employee
Splunk Employee

mark your own answer as accepted!

--
Jesse Trucks
Minister of Magic
0 Karma
Get Updates on the Splunk Community!

Now Available: Cisco Talos Threat Intelligence Integrations for Splunk Security Cloud ...

At .conf24, we shared that we were in the process of integrating Cisco Talos threat intelligence into Splunk ...

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Easily Improve Agent Saturation with the Splunk Add-on for OpenTelemetry Collector

Agent Saturation What and Whys In application performance monitoring, saturation is defined as the total load ...