topic Re: dedup by time in Getting Data In

dedup by time

gba8912 — Thu, 10 Dec 2020 17:51:16 GMT

hello,

I am trying to dedup events from successful authorizations in Splunk. Currently, our windows systems make about 4 events per authorization but we only want to see one. I would like to dedup based on time, 0.5 seconds for each event. Here is my current search:

| tstats summariesonly=true allow_old_summaries=true count from datamodel=Authentication.Authentication where Authentication.user=* (Authentication.src=* OR Authentication.dest=*) Authentication.action=failure by Authentication.user, Authentication.src, Authentication.dest

| rename "Authentication.*" as "*"

| eval source&destination=mvappend(src,dest)

| eventstats dc(source&destination) AS host_count by user

| where host_count >= 1

| sort - host_count

| table source&destination, user

| head 250

How can i add a dedup by time here?

Thanks!

Re: dedup by time

to4kawa — Fri, 11 Dec 2020 14:11:19 GMT

This result doesn't include the time.

>our windows systems make about 4 events per authorization
What are these?

Re: dedup by time

gba8912 — Fri, 11 Dec 2020 15:22:42 GMT

An example is event 4624, when this event is triggered in our DC, it make 4 events for that single sign on by a user or system. I don't know why this is but that's how its happening. We then get 4 events in Splunk. We made a dashboard with the above search and want to filter out the duplicates in order to view only one event per successful authorization.

I know time is not added, but we want to add it. I was hoping to add something that dedups events say within 0.1 seconds. This way it filters out 3 out of the 4 events that happened at the exact same time. Is this possible?

Re: dedup by time

to4kawa — Fri, 11 Dec 2020 23:37:57 GMT

I don't know, because I haven't looked at the logs.