topic How to extract timestamp without destroying the sequence of original events from my sample data? in Getting Data In

How to extract timestamp without destroying the sequence of original events from my sample data?

tac24 — Tue, 29 Sep 2020 20:01:31 GMT

Hi,

I have the following sample event data.
- For some reason, there is no sub-seconds-order data for the timestamp.
- Original event data does not have ts_SEQ field; I just added for the reference.

ts,                    ev_id,   val,      ts_SEQ
2018-6-17 08:00:01,     A,          10,       1 
2018-6-17 08:00:01,        B,            0,        2 
2018-6-17 08:00:01,     C,           3,       3
2018-6-17 08:00:11,     A,          20,          4
2018-6-17 08:00:11,     B,           0,          5
2018-6-17 08:00:11,     C,          -1,          6
2018-6-17 08:00:20,     A,           5,          7
2018-6-17 08:00:21,     B,           0,         8
2018-6-17 08:00:21,     C,          12,         9

What I want to do is to extract transactions; It seems that events A-B-C (ev_id) make one transaction group, almost every ten seconds. For example, transaction startswith=eval(ev_id="A") endswith=eval(ev_id="C") maxspan=2s could be applied.

When I indexed the above sample event data, at Set Source Type step, I chose Source Type=CSV, Timestamp Extraction=AUTO. What I got by spl source=... | table _time ts ev_id val ts_SEQ | sort _time is shown below.

Unfortunately, the ascending order of _time destroys the original event sequence (ts_SEQ), that means,
the above mentioned transaction extraction would be impossible. Currently, my work around is to add the ts_SEQ value to the original data before indexing, and use ts_SEQ to keep the original event sequence.

Question: How to extract timestamp without destroying the sequence of original events from my sample data?

Thank you.

Re: How to extract timestamp without destroying the sequence of original events from my sample data?

richgalloway — Mon, 18 Jun 2018 14:28:23 GMT

Splunk is a time-oriented service. That is, time sequence matters more than event sequence. See @dwaddle's answer at https://answers.splunk.com/answers/665477/retrievedownload-the-original-source-files-after-a.html#answer-665687 for an excellent explanation of this.

Re: How to extract timestamp without destroying the sequence of original events from my sample data?

tac24 — Mon, 18 Jun 2018 23:32:22 GMT

Thanks richgalloway,

That's what I want to find solutions.
In other words, at the indexing stage, if multiple events have same
timestamp value (let's assume YYYY-MM-DD HH:MM:SS format),
is there any way to keep the sequence of the original events
when table _time ... command is applied without assist data (ts_SEQ)?

tac24

Re: How to extract timestamp without destroying the sequence of original events from my sample data?

richgalloway — Tue, 19 Jun 2018 00:44:09 GMT

As dwaddle says, the only guarantee is time. Events with the exact same timestamp may or may not be returned in the same order in which they appear in the original file.