topic Re: remove duplicate or similar event in a trasaction command from the search in Splunk Search

remove duplicate or similar event in a trasaction command from the search

amir_thales — Tue, 29 Sep 2020 17:30:02 GMT

Hello Everybody,

I want to remove similar event which are in a transaction command.

In my case, I want to merge the eventcode 4663 similar so that only 1 eventcode 4663

Be careful, there are event code 4663 that are not similar so there will be 2 event code 4663 in this case.

Here is my request which display the result below:

host="XXXX" "eventcode=4663" OR "eventcode=7036" | transaction startswith="*running state." endswith="*stopped state."

i try dedup but without success.

Thank you
Amir

Re: remove duplicate or similar event in a trasaction command from the search

kamlesh_vaghela — Fri, 05 Jan 2018 16:16:00 GMT

Hi @amir_thales,

I'm uncleared about your requirement.

Meanwhile can you please try mvdedup?

host="XXXX" "eventcode=4663" OR "eventcode=7036" | transaction startswith="*running state." endswith="*stopped state." | eval eventcode=mvdedup(eventcode).

https://docs.splunk.com/Documentation/SplunkCloud/6.6.3/SearchReference/MultivalueEvalFunctions#mvdedup.28X.29

Thanks

Re: remove duplicate or similar event in a trasaction command from the search

amir_thales — Mon, 08 Jan 2018 09:54:48 GMT

Hello @kamlesh_vaghela and everybody,

The solution you proposed to me does not work.

I want to merge the same events which are between eventcode="7036", i want to merge all duplicates so that only one eventcode = "4663" remains.

But i want to do a difference between eventcode="4663" there are message where the eventcode is 4663 but the message is different and i want to merge duplicate and only display a message of each.

for example 1:

eventcode"7336"
eventcode"4663" -> message A
eventcode"4663" -> message A ---> here i want to merge eventcode"4663"so that there is only one left because these events are the same.
eventcode"4663" -> message A
eventcode"7336"

example 2:

eventcode"7336"
eventcode"4663" -> message A
eventcode"4663" -> message A ---> here i want to merge eventcode"4663"->message A so that there is only one left because these events are the same.
eventcode"4663" -> message B ---> here i want to remove one eventcode"4663"->message B so that there is only one left because these events are the same
eventcode"4663" -> message B
eventcode"7336"

thank you

Amir

Re: remove duplicate or similar event in a trasaction command from the search

kamlesh_vaghela — Mon, 08 Jan 2018 13:05:40 GMT

Hi @amir_thales,
Can you please share sample events?

Re: remove duplicate or similar event in a trasaction command from the search

amir_thales — Mon, 08 Jan 2018 13:13:30 GMT

@kamlesh_vaghela,

i put a sample in my first post.

i have 3 eventcode"4663" and i want to merge them.

Maybe, i must do something before to do the "transaction" but i don't know any function which merge similar events.

thank you

Re: remove duplicate or similar event in a trasaction command from the search

kamlesh_vaghela — Mon, 08 Jan 2018 13:57:47 GMT

Hi
Can you please try this ?

sourcetype="WinEventLog:Security"  "EventCode=4663" OR "EventCode=7036"
| rex field=_raw "EventCode=(?<EventRaw>.*)" max_match=0
| eval EventRaw=mvdedup(EventRaw) 
| table _time EventRaw

Re: remove duplicate or similar event in a trasaction command from the search

amir_thales — Mon, 08 Jan 2018 14:06:16 GMT

@kamlesh_vaghela,

This request display me a table which list all eventcode"4663".

So i want just merge the eventcode"4663" which are between the eventcode"7036".

I just read the different evencode'4663' and I noticed that the eventcodes "4663" were not identical because the application that executes is different.

So much for me and thank you for your help.

If you know a function or a way of answering the original question even if my problem is solved it would not be a refusal, so it will be useful for me in the future.

Thank you
Amir