Installation

How can I merge events based on Start and End value pairs when those values differ each time?

Pradeepkandukoo
Engager

How can I merge events based on Start and End value pairs when Start and end value differs each time?

For example: I have events as below.
Timestamp5 End Ticket: 2
Timestamp4 data3
Timestamp3 data2
Timestamp2 data1
Timestamp1 Start Ticket: 2
Timestamp5 End Ticket: 1
Timestamp4 data3
Timestamp3 data2
Timestamp2 data1
Timestamp1 Start Ticket: 1

So now i want group these events into one event based on the Ticket number in real time.

Can some one suggest how we can do it?

Tags (1)
0 Karma

solarboyz1
Builder

If there is a ticketID field in the events already, the easiest way is to use that field to join the events.

| transaction ticketID startswith="Start Ticket" endswith="End Ticket" 

However, there are limits to the transaction function.
You can also use stats, to similarly join:

| stats values(data1), values(data2), sum(data3), min(_time) as Start, max(_time) as Stop by ticketID

stats won't have the same limits as transaction.

0 Karma

woodcock
Esteemed Legend

Like this:

| makeresults | eval raw="Timestamp5 End Ticket: 2:::Timestamp4 data3:::Timestamp3 data2:::Timestamp2 data1:::Timestamp1 Start Ticket: 2:::Timestamp5 End Ticket: 1:::Timestamp4 data3:::Timestamp3 data2:::Timestamp2 data1:::Timestamp1 Start Ticket: 1"
| makemv delim=":::" raw
| mvexpand raw
| rename raw AS _raw
| fields - _time

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| streamstats count(eval(searchmatch("End Ticket"))) AS SessionID
| reverse
| list(_raw) AS events BY SessionID

shin_matsuzawa
Explorer

Hi,

I was surprised that there was such a way.
I'm interested in it.

I didn't know the "streamstats" command and I will try this command in my environment.

I appreciate your idea.
Thank you.

0 Karma

woodcock
Esteemed Legend

Don't forget to come back here to UpVote and click Accept if you get a good answer.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Ignoring the 'real time' part of the question, it seems a transaction will help. Try this

index = foo | transaction startswith="Start Ticket" endswith="End Ticket" | ...

Be aware that transaction is a slow command. Like @adonio said, if you can provide more information about your end goal, we may be able to offer a better answer.

---
If this reply helps you, Karma would be appreciated.
0 Karma

cpetterborg
SplunkTrust
SplunkTrust

As Rich says, "ignoring real time". You don't want to do a transaction command in a real-time search, it will just mess things up. Hopefully you are meaning that you want to just get the results at search time, not index time when you say "real time."

0 Karma

shin_matsuzawa
Explorer

Hi,

Do you mean that you want to put together the events with the same Ticket?
Do you have any element?

Following is an example:

2018-12-13 14:08:24,281 id-bbb End Ticket: 2
2018-12-13 14:07:24,281 id-bbb data3
2018-12-13 14:06:24,281 id-bbb data2
2018-12-13 14:05:24,281 id-bbb data1
2018-12-13 14:04:24,281 id-bbb Start Ticket: 2
2018-12-13 14:08:24,281 id-aaa End Ticket: 1
2018-12-13 14:07:24,281 id-aaa data3
2018-12-13 14:06:24,281 id-aaa data2
2018-12-13 14:05:24,281 id-aaa data1
2018-12-13 14:04:24,281 id-aaa Start Ticket: 1

If your events have elements like a "id-aaa", you can combine events using "transaction" command.

ex.

source="test.log" host="test01" sourcetype="test"
| rex field=_raw "(?ms)^[^,\\n]*,\\d+\\s+(?P<id>[^ ]+)"
| transaction host id startswith="Start" endswith="End"

Is my image correct?

alt text

adonio
Ultra Champion

please elaborate,
what is the problem you are trying to solve?
when you say "group these events to one event", what exactly do you mean?
what are you intend doing with that "group"?
do you need the events that have the start and end ticket as well?

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...