Getting Data In

How to extract timestamp without destroying the sequence of original events from my sample data?

tac24
New Member

Hi,

I have the following sample event data.
- For some reason, there is no sub-seconds-order data for the timestamp.
- Original event data does not have ts_SEQ field; I just added for the reference.

ts,                    ev_id,   val,      ts_SEQ
2018-6-17 08:00:01,     A,          10,       1 
2018-6-17 08:00:01,        B,            0,        2 
2018-6-17 08:00:01,     C,           3,       3
2018-6-17 08:00:11,     A,          20,          4
2018-6-17 08:00:11,     B,           0,          5
2018-6-17 08:00:11,     C,          -1,          6
2018-6-17 08:00:20,     A,           5,          7
2018-6-17 08:00:21,     B,           0,         8
2018-6-17 08:00:21,     C,          12,         9

What I want to do is to extract transactions; It seems that events A-B-C (ev_id) make one transaction group, almost every ten seconds. For example, transaction startswith=eval(ev_id="A") endswith=eval(ev_id="C") maxspan=2s could be applied.

When I indexed the above sample event data, at Set Source Type step, I chose Source Type=CSV, Timestamp Extraction=AUTO. What I got by spl source=... | table _time ts ev_id val ts_SEQ | sort _time is shown below.
alt text

Unfortunately, the ascending order of _time destroys the original event sequence (ts_SEQ), that means,
the above mentioned transaction extraction would be impossible. Currently, my work around is to add the ts_SEQ value to the original data before indexing, and use ts_SEQ to keep the original event sequence.

Question: How to extract timestamp without destroying the sequence of original events from my sample data?

Thank you.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Splunk is a time-oriented service. That is, time sequence matters more than event sequence. See @dwaddle's answer at https://answers.splunk.com/answers/665477/retrievedownload-the-original-source-files-after-a.html#an... for an excellent explanation of this.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

tac24
New Member

Thanks richgalloway,

That's what I want to find solutions.
In other words, at the indexing stage, if multiple events have same
timestamp value (let's assume YYYY-MM-DD HH:MM:SS format),
is there any way to keep the sequence of the original events
when table _time ... command is applied without assist data (ts_SEQ)?

tac24

0 Karma

richgalloway
SplunkTrust
SplunkTrust

As dwaddle says, the only guarantee is time. Events with the exact same timestamp may or may not be returned in the same order in which they appear in the original file.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.